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March  15,  1996 

MEMORANDUM  FOR  DIRECTOR,  DEFENSE  INTELLIGENCE  AGENCY 


SUBJECT:  Audit  Report  on  Agency  Resource  Management  Information  Support 
System  (Report  No.  96-085) 


We  are  providing  this  audit  report  for  review  and  comment.  The  audit  was 
performed  in  response  to  a  request  from  Senator  Strom  Thurmond.  The  Office  of 
Audit,  Inspector  General,  Defense  Intelligence  Agency,  assisted  in  performing  the 
audit.  Management  comments  on  a  draft  of  this  report  were  considered  in  preparing 
the  final  report. 

DoD  Directive  7650.3  requires  that  all  unresolved  issues  be  resolved  promptly. 
Management  comments  on  the  draft  report  did  not  discuss  the  specific  recommended 
actions.  Therefore,  we  request  that  the  Defense  Intelligence  Agency  provide  additional 
comments  by  May  14,  1996,  on  the  development  of  a  comprehensive  acquisition  plan 
for  the  Agency  Resource  Management  Information  Support  System;  establishment  of 
controls  to  verify  documentation  of  intended  users  requirements;  establishment  of 
controls  to  verify  that  procurement  quantities  are  limited  until  suitability  is 
demonstrated;  and  establishment  of  controls  to  ensure  that  senior  managers  are  kept 
apprised  of  important  cost,  schedule,  and  performance  data.  See  the  Finding  for  the 
required  responses. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the 
audit  should  be  directed  to  Mr.  Harlan  M.  Geyer,  Audit  Program  Director,  at 
(703)  604-9594  (DSN  664-9594)  or  Ms.  Jenniffer  Wilson,  Audit  Project  Manager,  at 
(703)  604-8361  (DSN  664-8361).  See  Appendix  G  for  the  report  distribution.  Audit 
team  members  are  listed  inside  die  back  cover. 


Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 
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Executive  Summary 


Introduction.  This  audit  was  performed  in  response  to  a  request  from  Senator  Strom 
Thurmond  to  review  allegations  of  improprieties  related  to  the  procurement  of 
commercial  software  for  the  Agency  Resource  Management  Information  Support 
System.  The  Defense  Intelligence  Agency  intended  for  the  Agency  Resource 
Management  Information  Support  System  to  be  an  integrated  suite  of  commercial 
computer  software  that  would  standardize  existing  administrative  systems,  eliminate 
redundant  data  bases  and  rekeying  of  data,  promote  the  exchange  of  information,  and 
provide  a  common  data  base  of  resource  information  to  support  management.  In  June 
1993,  the  Defense  Intelligence  Agency  awarded  a  contract,  totaling  about  $6.9  million, 
to  acquire  and  customize  commercial  software  to  fit  Defense  Intelligence  Agency 
business  practices.  In  September  1993  and  1994,  the  Defense  Intelligence  Agency 
issued  two  delivery  orders  totaling  $1.1  million  under  an  existing  indefinite-delivery, 
indefinite-quantity  contract  to  integrate  the  commercial  software  modules  that  were 
procured  under  the  Agency  Resource  Management  Information  Support  System 
contract.  The  National  Military  Intelligence  Systems  Center,  Defense  Intelligence 
Agency,  planned  to  use  the  software  modules  to  replace  systems  in  two  functions. 

Audit  Objectives.  The  audit  objectives  were  to  determine  whether  the  Defense 
Intelligence  Agency  management  of  the  Agency  Resource  Management  and  Information 
Support  System  and  actions  taken  to  correct  system  deficiencies  conformed  to 
regulatory  guidance  and  to  evaluate  the  validity  of  the  allegations.  We  also  evaluated 
the  Defense  Intelligence  Agency  management  control  program  as  it  related  to  the  audit 
objectives. 

Audit  Results.  The  audit  either  substantiated  or  partially  substantiated  six  allegations 
related  to  inadequate  planning  and  management  of  the  acquisition  and  implementation 
of  the  Agency  Resource  Management  Information  Support  System.  The  Defense 
Intelligence  Agency  spent  more  than  $5.1  million  for  the  commercial  software  and 
contractor  services  that  did  not  satisfy  the  requirement  for  an  integrated,  management 
information  system.  We  did  not  substantiate  two  other  allegations  (see  Appendix  C). 

The  Defense  Intelligence  Agency  management  control  program  needs  improvement 
because  material  weaknesses  were  identified  related  to  acquisition  planning  and 
execution.  Recommendations,  if  implemented,  will  improve  the  effectiveness  of 
actions  to  develop  an  integrated,  management  support  system  and  should  reduce  future 
acquisition  costs.  We  could  not  quantify  potential  monetary  benefits.  See  Part  I  for  a 
discussion  of  the  audit  results  and  Appendix  E  for  a  summary  of  the  potential  benefits 
resulting  from  audit. 
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Summary  of  Recommendations.  We  recommend  stopping  all  Agency  Resource 
Management  Information  Support  system  development  and  implementation  until  a 
comprehensive  acquisition  plan  is  developed  and  management  controls  are  established 
to  ensure  that  future  software  acquisition  and  development  efforts  are  subjected  to 
rigorous  senior  management  oversight. 

Management  Comments.  The  Defense  Intelligence  Agency  concurred  with  all 
recommendations;  however,  the  comments  did  not  fully  meet  our  intent.  See  Part  I  for 
a  summary  of  management  comments  regarding  the  recommendations,  Appendix  D  for 
a  summary  of  specific  management  comments,  and  Part  III  for  the  complete  text  of 
management  comments. 

Audit  Response.  The  management  comments  did  not  specifically  discuss  the 
recommended  actions.  In  addition,  the  Defense  Intelligence  Agency  nonconcurred  with 
specific  issues  discussed  in  the  finding.  Management  made  comments  concerning  the 
acquisition  strategy  for  procuring  the  commercial-off-the-shelf  software  products, 
demonstration  and  prototyping  of  the  software  products,  substantiation  of  the 
allegations,  the  Agency  Resource  Management  Information  Support  System  concept  of 
operations  and  requirements,  and  program  management  roles  and  responsibilities.  We 
disagreed  with  those  comments  and  provided  a  response  in  Appendix  D  of  this  report. 
Therefore,  we  ask  that  the  Defense  Intelligence  Agency  provide  additional  comments  in 
response  to  this  report  by  May  14,  1996. 
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Part  I  -  Audit  Results 


Audit  Results 


Audit  Background 


Need  for  an  Improved  Administrative  Management  Information 
System.  Defense  Intelligence  Agency  (DIA)  administrative  systems  are 
fragmented,  labor  intensive,  and  inefficient.  In  general,  administrative 
information  is  redundantly  processed  in  stand-alone  systems  that  do  not  provide 
common  access.  In  July  1988,  the  Director,  DIA,  approved  a  concept  for 
improving  the  management  of  administrative  information  at  the  corporate  level 
through  an  open  systems  architecture.  Working  groups  and  a  process  action 
team  were  formed  to  identify  DIA-wide  applications  for  integration  into  the 
DIA  management  information  system.  To  meet  the  need  for  an  improved 
administrative  management  system,  the  National  Military  Intelligence  Systems 
Center  (the  Systems  Center)  developed  the  concept  for  the  Agency  Resource 
Management  Information  Support  System  (ARMISS).  In  July  1988,  the  Deputy 
Director,  DIA,  directed  the  Systems  Center  to  establish  a  program  management 
office  for  implementing  the  new  administrative  system.  The  Chief  of  Logistics 
Division  within  the  Systems  Center  was  designated  the  ARMISS  program 
manager.1 

ARMISS  Concept  of  Operations.  DIA  planned  to  replace  its  stand-alone 
automated  information  systems  with  one  integrated,  information  management 
system.  The  goal  was  consistent  with  Corporate  Information  Management 
policies  and  principles,  as  defined  in  DoD  Directive  8000.1,  "Defense 
Information  Management  Program,"  October  27,  1992,  which  emphasized  the 
capability  of  automated  data  processing  in  support  of  business  process 
improvements,  data  standardization,  and  systems  consolidation.  Principles 
associated  with  meeting  the  Corporate  Information  Management  goals  include 
functional  process  improvement  projects  and  functional  and  technical  integration 
analysis  and  planning; 

ARMISS  was  intended  to  promote  exchanging  and  sharing  information, 
minimizing  redundant  development  of  user  applications,  reducing  the  potential 
for  errors  by  eliminating  multiple  data  entries,  maximizing  data  availability  to 
upper  management  for  decision  making,  and  permitting  electronic  tracking  of  a 
variety  of  data.  The  ARMISS  would  encompass  all  aspects  of  management  and 
administrative  information  used  by  mid-level  and  senior  managers  to  make 
decisions  on  administrative  and  fiscal  matters.  The  functional  offices  included 
in  the  ARMISS  effort  consists  of  the  comptroller,  contracts,  logistics, 
personnel,  training,  information  processing,  security,  legal,  and  administrative 
functions  of  all  DIA  elements.  The  ARMISS  program  manager  estimated  that 


!The  term  "ARMISS  program  manager,"  used  throughout  this  report,  refers  to 
the  Chief  of  the  Logistics  Division  who  served  in  many  different  roles  and 
positions  related  to  the  ARMISS  effort.  The  program  manager  was  the  overall 
leader  of  the  planning  effort  that  set  the  direction  and  strategy  for  the  ARMISS 
project.  In  that  capacity,  the  program  manager  held  titles  and  responsibilities  of 
the  acquisition  planning  leader,  the  project  management  and  oversight  officer, 
the  technical  process  improvements  process  action  officer,  the  Co-chair  of  the 
Administrative  Functional  Control  Board,  and  the  DIA-perspective  focal  point. 
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commercial  software  would  give  DIA  a  prepackaged  solution  to  satisfy  the 
predominant  (about  80  percent)  ARMISS  requirements.  The  DIA  spends  about 
$4.1  million  annually  for  new  commercial  software. 

Contracting  for  Acquisition  of  the  ARMISS.  On  June  7,  1993,  the  DIA 
Virginia  Contracting  Activity  awarded  contract  MDA908-93-C-0027  to  the 
Value  System  Engineering  Corporation  (VSE)  to  procure  and  customize  an 
integrated  commercial-off-the-shelf  (COTS)  software  suite  to  serve  as  the 
ARMISS.  The  VSE  based  its  bid  on  using  Oracle  Corporation  COTS  software 
products  for  most  functions.  The  Oracle  COTS  software  products  consisted  of  a 
budget,  acquisition,  personnel  and  logistics  module.  The  ARMISS  effort  was  a 
cost-plus-fixed-fee-completion  type  contract  for  procurement  and  customization 
of  COTS  software  and  a  firm  fixed-price  type  contract  for  software 
maintenance.  The  contract  period  was  4  years  and  4  months  (a  base  period  of 
16  months  and  3  option  years)  for  a  total  price  of  $6.9  million.  Implementation 
of  the  ARMISS  was  unsuccessful;  therefore,  DIA  allowed  the  contract  to  expire 
and  exercised  no  options.  The  DIA  paid  VSE  about  $4.1  million  for  software 
and  services. 

The  DIA  is  continuing  efforts  to  upgrade  its  existing  administrative  information 
systems  through  the  use  of  software  acquired  under  the  ARMISS  contract.  The 
DIA  paid  $568,408  for  delivery  order  10  and  $578,430  for  delivery  order  30. 
Those  delivery  orders  were  issued  on  September  23,  1993,  and  September  22, 
1994,  respectively,  to  the  Computer  Sciences  Corporation  (CSC)  under  contract 
MDA908-93-D-1503,  an  indefinite-delivery,  indefinite-quantity  contract.  The 
Systems  Center  used  delivery  orders  10  and  30  to  implement  the  ARMISS 
software  to  satisfy  deficiencies  for  two  functional  missions  within  the  Systems 
Center.  The  delivery  order  efforts  are  completed,  yet  the  concept  of  an 
integrated,  information  management  system  was  not  achieved. 

Allegations  Related  to  the  Procurement  of  ARMISS.  On  January  3,  1995, 
the  Chairman,  Senate  Committee  on  Armed  Services  requested  that  the 
Inspector  General,  DoD,  investigate  allegations  of  improprieties  by  the  DIA  in 
procuring  ARMISS  software.  The  specific  allegations  and  the  results  of  our 
audit  pertaining  to  each  allegation  are  in  Appendix  C. 


Audit  Objectives 


The  audit  objectives  were  to  determine  whether  the  DIA  management  of  the 
acquisition  of  the  ARMISS  and  actions  to  correct  system  deficiencies  conformed 
to  regulatory  guidance  and  to  evaluate  the  validity  of  the  allegations.  We  also 
evaluated  management  controls  germane  to  the  audit  objectives.  See  the 
Finding  for  a  discussion  of  the  material  management  control  weaknesses  and 
Appendix  A  for  a  discussion  of  the  audit  scope  and  methodology  and  the  review 
of  the  management  control  program.  See  Appendix  B  for  a  summary  of  prior 
coverage  related  to  the  audit  objectives. 
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The  DIA  did  not  exercise  effective  control  over  efforts  to  acquire, 
customize,  and  integrate  COTS  software  for  the  ARMISS.  Specifically, 
DIA: 


o  procured  100  percent  of  the  assumed  quantities  of  COTS 
software  needed  for  the  ARMISS  without  verifying  that  the  system 
concept  design  was  feasible; 

o  did  not  adequately  define  ARMISS  requirements  before 
contract  award; 

o  did  not  document  business  practices  to  facilitate  customization 
of  the  COTS  software; 

o  did  not  consider  the  requirement  to  rely  on  the  National 
Security  Agency  (NSA)  for  financial  and  accounting  services;  and 

o  continued  development  efforts  after  it  was  apparent  that  the 
acquired  COTS  software  was  not  suitable  for  the  ARMISS  concept. 

Consequently,  DIA  spent  more  than  $5.1  million  for  COTS  software  and 
contractor  customization  and  integration  services  that  did  not  result  in  an 
integrated,  management  information  system  or  improved  capability  to 
DIA- wide  business  processes. 


Acquisition  Planning  for  the  ARMISS 


DIA  Time-Phased  Replacement  of  Existing  Systems.  The  ARMISS  request 
for  proposal,  dated  July  17,  1992,  states  that  the  systems  and  applications 
integral  to  DIA  operations  must  remain  operational  and  would  not  be  replaced 
by  ARMISS.  Thus,  to  ensure  continuity  of  operations,  implementation  of  the 
ARMISS  would  require  maintenance  and  operation  of  dual  systems  and  the 
development  of  interfaces  to  external  systems.  The  DIA  planned  to  implement 
the  ARMISS  in  the  following  functional  modules: 

o  budget, 

o  acquisition, 

o  personnel, 

o  training  and  education, 

o  logistics. 
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o  travel, 

o  central  repository,  and 

o  executive  mail. 

The  DIA  planned  to  award  one  contract  to  procure  and  customize  the  COTS 
software.  The  DIA  also  planned  to  integrate  the  software  into  the  existing  DIA 
environment  using  the  DoD  Intelligence  Information  Systems  Integration  and 
Engineering  Support  Contract  (DIESCON). 

Prototyping  as  a  Means  to  Reduce  Risk  and  Uncertainty  in  the 
ARMISS.  DoD  Directive  8120.1,  "Life-Cycle  Management  of  Automated 
Information  Systems,"  January  14,  1993,  requires  that  DoD  Components 
consider  the  acceptance  of  software  based  on  approval  of  test  results.  DoD 
Components  are  encouraged  to  use  the  policies  and  procedures  identified  in 
DoD  Directive  5000.2,  "Defense  Acquisition  Management  Policies  and 
Procedures,"  February  23,  1991,  for  managing  all  acquisition  programs-major 
and  nonmajor  programs.  In  accordance  with  DoD  Directive  5000.2,  part  5, 
section  D,  "Technology  Transition  and  Prototyping,"  prototyping  is  a  major 
element  of  the  acquisition  process  and  should  be  used  during  the  concept 
demonstration  phase  of  the  acquisition  to  reduce  risk  and  uncertainty  associated 
with  the  integration  of  products  into  system  concept  designs.  Testing  of 
prototypes  demonstrates  the  feasibility  of  the  product  to  work  in  the  system 
design  and  allows  the  early  assessment  of  operational  effectiveness  and 
suitability  of  the  product. 

To  achieve  the  economies  of  a  bulk  purchase,  DIA  elected  to  procure  the  Oracle 
software  immediately  after  the  contract  was  awarded  to  VSE  on  June  7,  1993. 
DIA  procured  Oracle  products  and  tools  and  site  licenses  for  500  concurrent 
users  even  though  the  concept  of  integrating  all  the  software  applications  into 
the  DIA  environment  had  not  been  demonstrated.  Further,  the  ARMISS 
contract  required  VSE  to  customize  the  software  applications  over  a  12-  to 
24-month  period;  therefore,  there  was  ample  time  to  prove  the  feasibility  of  the 
ARMISS  concept  using  a  limited  number  of  Oracle  software  modules  and  site 
licenses  in  a  prototype  installation.  A  more  cost-effective  acquisition  alternative 
would  have  been  for  DIA  to  procure  the  needed  quantities  of  software  in  a  bulk 
purchase  after  DIA  demonstrated  the  feasibility  of  the  ARMISS  concept. 


Technical  Constraints  Affecting  ARMISS  Implementation 


Documentation  of  Business  Processes.  DoD  Directive  8120.1  implements 
Corporate  Information  Management  policies  and  principles  related  to  the 
acquisition  of  automated  information  systems.  The  Directive  states  that  it  is 
DoD  policy  to  use  life-cycle  management  review  and  approval  procedures  to 
verify  that  programmatic  decisions  for  all  automated  information  systems  are 
based  on  approved  functional  requirements  analysis  and  strategic  planning. 
Specifically,  DoD  Components  are  responsible  for  developing  a  current  and 
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future  baseline  of  the  administrative  processes  in  the  early  stages  of  the 
acquisition  life-cycle  for  all  automated  information  system  projects. 
Development  of  the  current  and  future  baseline  is  accomplished  by  preparing 
detailed  definitions  of  functional  requirements  for  all  functional  areas  and  by 
producing  a  business  plan  that  identifies  changes  in  the  business  process  and  an 
implementation  strategy  for  the  new  system. 

The  DIA  Manual  44-2,  "Acquisition,"  March  1993,  requires  acquisition  plans 
to  identify  significant  cost  and  technical  or  schedule  constraints  that  affect  the 
acquisition.  The  ARMISS  program  manager  was  responsible  for  determining 
and  validating  requirements,  acquisition  planning,  system  concept  design,  the 
implementation  plan,  and  funding  for  the  ARMISS  project.  DIA  ensured  that 
the  ARMISS  contract  required  minimal  customization  of  COTS  software, 
mature  and  well-documented  Government  business  processes,  and  a  defined 
business  model  of  the  operations  of  each  functional  office  obtainable  within 
30  to  35  days.  Documentation  of  the  supported  business  processes  was 
necessary  to  permit  customization  the  COTS  software.  However,  the  ARMISS 
program  manager  did  not  require  DIA  functional  managers  to  document  detailed 
definitions  of  their  functional  requirements  and  business  processes.  The 
ARMISS  program  manager  did  not  request  the  detailed  documentation  because 
he  assumed: 

o  that  the  COTS  software  would  provide  an  80-percent  solution  for 
processing  requirements,  thereby  resulting  in  a  rapid  prototyping  approach 
rather  than  a  software  development;  and 

o  that  DIA  would  change  its  way  of  doing  business  to  take  advantage  of 
the  COTS  software  capabilities  and  efficiencies. 

Effects  of  DIA  Undefined  Requirements  on  VSE  Ability  to  Perform.  As  a 
result  of  the  inadequate  requirements  documentation,  DIA  had  no  assurances 
that  the  Oracle  software  would  satisfy  the  ARMISS  concept.  Further,  because 
of  undefined  requirements,  VSE  was  unable  to  customize  the  Oracle  software  to 
the  DIA  business  processes.  By  early  August  1993,  VSE  determined  that 
detailed  documentation  of  the  DIA  current  and  future  operational  requirements 
was  lacking.  VSE  needed  the  documentation  to  customize  the  software  for  the 
Oracle  personnel  and  acquisition  modules,  as  planned  for  during  the  base  period 
of  the  contract.  DIA  modified  the  contract  on  September  23,  1993,  and  again 
on  March  31,  1994,  to  direct  VSE  to  develop  the  necessary  requirements 
documentation.  VSE  delivered  a  Functional  Requirements  Definition  Document 
for  the  Oracle  acquisition  module  on  May  23,  1994,  and  for  the  Oracle 
personnel  module  on  September  21,  1994,  at  a  total  cost  to  DIA  of  about 
$928,000. 

Operational  Analysis  of  the  Oracle  Acquisition  Module.  VSE 
performed  operational  analyses  of  the  Oracle  acquisition  module  from  February 
through  December  1993.  The  documentation  provided  a  detailed  definition  of 
the  operational  requirements  for  the  DIA  acquisition  process  and  a  business 
model  for  determining  current  and  future  needs  for  ARMISS.  As  a  result  of  the 
analysis,  DIA  determined  that  the  Oracle  acquisition  module  did  not  conform  to 
the  DIA  acquisition  process  and  that  development  of  an  import  interface  to 
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external  procurement  and  logistics  systems  was  needed  to  provide  an  integrated 
capability.  Oracle  COTS  software,  however,  was  not  designed  to  interface  with 
external  systems,  and  software  modifications  needed  to  support  an  interface 
could  invalidate  software  warranties  and  maintenance  support  agreements. 

Operational  Analysis  of  the  Oracle  Personnel  Module.  VSE 
performed  an  operational  analysis  on  the  Oracle  personnel  module  from 
September  1993  through  September  1994.  DIA  subject  matter  experts  spent 
about  2,800  hours  working  with  Oracle  consultants  and  VSE  program  analysts 
to  document  the  detailed  operational  requirements  and  business  model  of  the 
current  and  future  needs  for  the  DIA  personnel  function. 

As  a  result  of  the  operational  analysis,  DIA  determined  that  the  Oracle 
personnel  software  module  did  not  provide  a  prepackaged  solution  to  satisfying 
the  Directorate  of  Human  Resources'  requirements  and  that  customization  of  the 
software  was  significantly  more  complex  than  envisioned.  The  Oracle 
personnel  module  did  not  offer  a  prepackaged  solution  because  it  did  not  contain 
forms  or  critical  data  fields  needed  to  process  mandatory,  Government-specific 
personnel  actions.  Customizing  the  software  required  extensive  programming, 
which  could  invalidate  the  software  maintenance  support  and  warranty 
coverage.  Because  the  Oracle  personnel  module  could  not  satisfy  DIA 
requirements,  the  Directorate  of  Human  Resources  began  evaluating  DoD- 
standard  systems,  in  accordance  with  Corporate  Information  Management 
policies  and  principles,  to  satisfy  deficiencies  in  its  existing  personnel  systems. 

Oracle  Software  Suitability  to  ARMISS  Requirements.  The  DIA  stated  that 
COTS  software  would  meet  80  percent  of  DIA  functional  requirements,  that 
DIA  was  willing  to  modify  its  business  processes  to  take  advantage  of 
commercial  software  capabilities  and  efficiencies,  and  that  DIA  did  not  desire  to 
modify  the  software  to  the  extent  DIA  would  have  to  develop  nonstandard 
programming  script  to  facilitate  unique  DIA  requirements.  However,  as  early 
as  August  1993,  VSE  determined  that  those  assumptions  were  erroneous  and 
that  DIA  did  not  have  an  adequate  understanding  of  the  complexities  associated 
with  using  industry-standard  commercial  software  as  a  solution  to  satisfying 
ARMISS  requirements. 

The  DIA  did  not  adequately  evaluate  the  mechanics  of  how  the  Oracle  software 
worked  and  the  effects  it  had  on  the  ARMISS  concept.  Specifically,  the 
ARMISS  contract  statement  of  work  required  that  ARMISS  interface  with 
external  finance,  logistics,  and  procurement  systems  in  order  to  import  data  to 
the  Oracle  applications.  However,  the  Oracle  software  does  not  have  a  standard 
interface  for  importing  data  from  external  systems.  According  to  the  Oracle 
Corporation,  there  are  limitations  and  explicit  warnings  against  importing  data 
in  non-Oracle  systems  to  Oracle  software.  Import  interface  is  considered  high 
risk  because  it  requires  development  of  an  import  facility  and  modification  to 
the  structure  of  the  Oracle  software.  Because  the  Oracle  Corporation  often 
changes  the  structure  each  time  it  releases  a  new  version  of  its  software, 
programming  scripts,  developed  to  support  the  import  facility,  may  not  function 
properly  and  may  have  to  be  modified  with  each  new  release  of  the  Oracle 
software.  Further,  modification  to  the  software  programs'  structures  would 
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require  an  effort  far  greater  than  that  expected  under  a  statement  of  work  that 
emphasized  that  DIA  did  not  wish  to  modify  the  commercial  software 
applications  to  facilitate  unique  DIA  requirements. 


Compatibility  of  System  Concept  Design  with  the  NSA 
Finance  and  Accounting  System 


Importance  of  General  Ledger  to  Operation.  The  Oracle  Government 
Financial  applications  includes  the  budget,  acquisition,  and  logistics  modules. 
The  general  ledger  component  of  the  budget  module  is  the  central  hub  that 
facilitates  the  exchange  of  data  among  the  Oracle  Financial  applications.  The 
Oracle  software  is  designed  as  a  tightly  integrated  software  system  that  assumes 
functional  managers  use  the  Oracle  budget  module  as  the  core  module  to 
exchange  data.  If  the  general  ledger  component  of  the  budget  module  is  not 
implemented,  the  capabilities  of  the  software  modules  are  reduced. 

Requirement  to  Use  the  NSA  System.  The  DIA  awarded  the  ARMISS 
contract  without  considering  that  the  Oracle  budget  module  may  duplicate  or 
conflict  with  functions  inherent  in  the  NSA  finance  and  accounting  system.  In 
August  1992,  the  Acting  Comptroller  of  the  DoD  (now  the  Under  Secretary  of 
Defense  [Comptroller])  directed  the  transfer  of  responsibility  for  DIA  finance 
and  accounting  functions  from  the  Air  Force  Finance  and  Accounting  Office, 
Bolling  Air  Force  Base,  to  the  NSA.  On  September  16,  1993,  the  Comptrollers 
of  the  DIA  and  the  NSA  signed  a  memorandum  of  understanding  covering  DIA 
use  of  the  NSA  finance  and  accounting  system.  The  Office  of  the  DIA 
Comptroller  advised  senior  DIA  management  and  the  ARMISS  program 
manager  that  the  NSA  would  provide  finance  and  accounting  support  at  least 
6  months  before  the  award  of  the  ARMISS  contract.  Nonetheless,  the  ARMISS 
contract  statement  of  work  had  no  requirement  that  DIA  use  the  NSA 
accounting  and  finance  system  or  that  an  interface  with  the  NSA  system  would 
be  needed. 

By  late  August  1993,  the  DIA  Comptroller  informed  the  Systems  Center  and 
other  DIA  functional  elements  that  he  objected  to  the  use  of  the  Oracle  budget 
module  as  the  core  to  provide  common  access  to  financial  data  because  use  of 
the  general  ledger  component  of  the  budget  module  would  unnecessarily 
duplicate  the  accounting  functions  performed  by  the  NSA  finance  and 
accounting  system.  At  the  request  of  the  ARMISS  contracting  officer 
representative,  VSE  performed  an  assessment  to  determine  the  effects  that  the 
transfer  of  accounting  and  finance  functions  had  on  the  ARMISS  concept  of 
operations.  VSE  delivered  the  assessment  in  October  1993  at  a  cost  of  $21,110. 
VSE  determined  that  the  implementation  of  accounting  and  finance  functions  in 
the  NSA  system  would  force  a  major  revision  to  the  ARMISS  concept. 
Specifically,  ARMISS  would  not  provide  a  real-time  exchange  of  financial  data 
between  the  two  systems,  would  not  eliminate  redundant  data  bases  and 
rekeying  of  data,  and  would  not  provide  a  single,  integrated  solution  as  DIA 
envisioned. 
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Ability  of  the  ARMISS  Contract  to  Meet  Goals 


Effects  of  Allowing  the  ARMISS  Contract  with  VSE  to  Expire.  After 
spending  about  $1  million  on  requirements  documentation,  DIA  terminated 
customization  of  the  Oracle  software  applications  on  March  31,  1994.  In 
August  1993,  DIA  redirected  VSE  efforts  under  the  ARMISS  contract  from 
software  customization  toward  functional  requirements  analysis.  Funds  that 
were  identified  in  the  base  year  of  the  contract  for  software  customization  were 
used  to  support  the  operational  analyses  that  VSE  performed  on  the  DIA 
acquisition  and  personnel  processes.  During  the  operational  analyses,  DIA 
determined  that  the  proposed  Oracle  commercial  software  solution,  with  limited 
customization,  would  not  satisfy  the  ARMISS  goals  for  an  integrated  business 
solution  tailored  to  current  DIA  processes.  The  Oracle  software  product  was 
not  designed  to  interface  with  external  systems,  such  as  those  in  the  DIA 
automated  environment.  Further,  the  Oracle  software  applications  were 
dependent  on  an  integral  general  ledger,  which  conflicted  with  the  requirement 
that  DIA  use  the  NSA  accounting  and  finance  system.  The  VSE  did  not  deliver 
the  customized  software  because  the  contractual  effort  escalated  beyond  the 
scope  of  work.  As  a  result,  DIA  allowed  the  contract  to  expire  without 
exercising  any  of  the  option  years.  The  full  value  of  the  ARMISS  contract 
totaled  $4.1  million. 


Project  Tracking  and  Oversight 


Roles  and  Responsibilities.  DIA  Regulation  65-17,  "Automated  Information 
Systems  (AIS)  Management  Policy,"  November  6,  1989,  assigns  the  Systems 
Center  overall  responsibility  for  the  development  and  maintenance  of  corporate, 
automated  information  systems.  Therefore,  the  Systems  Center  was  responsible 
for  the  implementation  of  the  ARMISS  into  the  DIA  automated  environment. 
That  responsibility  was  shared  between  the  offices  of  the  Logistics  Division  and 
Systems  Development.  The  Chief  of  the  Logistics  Division  was  the  ARMISS 
program  manager.  The  ARMISS  program  manager  had  overall  responsibility 
for  the  acquisition  planning,  to  include  determining  and  validating  the  ARMISS 
requirements,  designing  the  system  concept,  developing  the  implementation 
strategy,  and  managing  funds  and  resources  for  the  ARMISS  contract.  The 
DIA  contracting  officer  appointed  a  technical  manager  in  the  Systems 
Development  office  as  the  ARMISS  contracting  officer  representative.  The 
contracting  officer  representative  was  responsible  for  monitoring  contractor 
performance  and  for  representing  the  contracting  officer  in  all  technical  matters 
related  to  the  contract.  However,  cognizant  officials  at  VSE  and  at  the  DIA 
contracting  activity  indicated  that  the  ARMISS  program  manager  directed  VSE 
on  how  to  proceed  with  the  ARMISS  contract. 

In  September  1993,  VSE  complained  to  the  DIA  Director  of  Procurement  about 
the  potential  conflict  in  receiving  direction  from  multiple  sources.  In  November 
1993,  the  DIA  contracting  officer  issued  a  letter  to  VSE  reiterating  that  the 
contractor  shall  not  take  orders  to  perform  work  regarding  contract 
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MDA908-93-C-0027  from  anyone  except  the  contracting  officer  or  contracting 
officer  representative.  Thereafter,  the  ARMISS  program  manager  no  longer 
intervened  and  focused  on  implementing  and  integrating  an  ARMISS-like 
system  to  satisfy  Systems  Center  requirements.  The  ARMISS  program  manager 
initiated  two  delivery  orders  with  CSC,  under  the  DIESCON,  to  implement  and 
integrate  the  Oracle  commercial  software  and  Oracle  relational  data  base 
management  system.  The  ARMISS  program  manager  made  no  overall  plan  for 
tracking  the  project  or  for  reporting  the  status  of  the  ARMISS  or  DIESCON 
efforts  to  senior  DIA  management. 

DIA  Senior  Management  Briefings.  From  the  time  the  ARMISS  contract  was 
awarded  in  June  1993  through  the  time  the  contract  was  allowed  to  expire  in 
September  1994,  the  Systems  Center  held  only  two  meetings  with  senior  DIA 
management  concerning  the  status  of  the  ARMISS  project.2  The  ARMISS 
program  manager  initiated  the  delivery  orders  without  proper  planning 
documentation  and  senior  management  oversight.  In  our  opinion,  proper 
planning  did  not  occur  because  by  the  time  delivery  order  10  was  initiated  in 
September  1993,  the  DIA  Comptroller  had  reiterated  that  the  ARMISS  could 
not  be  the  final  authority  on  obligations  or  expenditures  and  could  not  provide 
an  integrated,  management  information  system  to  satisfy  DIA-wide  needs. 
Also,  by  the  time  the  Systems  Center  initiated  delivery  order  30  in  September 
1994,  VSE  had  determined  that  the  Oracle  software  did  not  conform  to  DIA 
business  processes  and  that  significant  risks  were  associated  with  required 
interfaces  with  external  systems. 

The  original  intent  of  using  DIESCON  was  for  installation  and  integration  of  the 
customized  software  into  the  DIA  automated  environment.  However,  VSE  was 
unable  to  customize  the  software  to  fit  DIA  business  processes.  Because  the 
ARMISS  program  manager  used  the  DIESCON  efforts  as  a  prototyping 
approach  to  satisfying  ARMISS  requirements  in  the  Systems  Center,  the 
Systems  Center  should  have  briefed  senior  DIA  management  on  the  purpose  of 
additional  expenditures  of  about  $1.1  million.  The  Systems  Center  did  not  brief 
DIA  senior  management  on  the  delivery  orders  until  March  1995. 

DIESCON  Delivery  Order  10.  On  September  23,  1993,  the  DIA 
contracting  activity  issued  delivery  order  10,  at  the  request  of  the  Systems 
Center.  The  delivery  order  was  a  firm  fixed-price-completion  effort  totaling 
$568,408.  The  scope  of  the  delivery  order  was  focused  on  implementing  a 
Systems  Center  Administrative  Management  System,  using  the  Oracle  data  base 
system  and  software  modules  acquired  under  the  ARMISS  contract,  in  a  test-bed 
environment,  for  proving  the  ARMISS  concept.  The  contractor  for  DIESCON, 
CSC,  delivered  an  Administrative  Management  System  application  framework 
to  include  the  data  base  schema,  data  dictionaries,  and  data  screens  supporting 


2The  Systems  Center  discussed  the  status  of  ARMISS  at  two  DIA  Executive 
Committee  meetings-one  on  August  3,  1992,  and  the  other  on  March  2,  1993. 
However,  the  role  and  function  of  the  DIA  Executive  Committee  was  only  to 
work  with  senior  managers  to  resolve  questions  and  concerns  and  to  provide  a 
forum  for  discussing  significant  internal  DIA  management  and  resource  issues. 
The  Committee  did  not  provide  an  oversight  role  or  enforce  decisions. 
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the  budget  functions.  The  CSC  also  delivered  an  Administrative  Management 
System  capability  on  a  stand-alone  computer  workstation  within  the  Logistics 
Division  of  the  Systems  Center  to  replace  the  current  applications  operating  on 
its  WANG  Office  System  and  local  area  network  environments  with  the  Oracle 
budget  module.  However,  the  ARMISS  concept  continued  to  be  invalid  because 
the  Oracle  budget  module  needed  to  interface  with  the  NSA  system  to  provide 
an  integrated  solution. 

Delivery  Order  30.  On  September  22,  1994,  the  DIA  contracting 
activity  issued  delivery  order  30  to  CSC  at  the  request  of  the  Systems  Center. 
The  delivery  order  was  a  firm  fixed-price-completion  effort  totaling  $578,430. 
The  objectives  of  delivery  order  30  were  to  migrate  existing  software 
applications  to  the  Oracle  relational  data  base  management  system  and  to 
improve  the  Systems  Center  acquisition  process  by  replacing  its  Acquisition, 
Resource,  Inventory,  Management  Information  System  (Acquisition  System). 
Delivery  order  30  was  to  support  continuation  of  the  Systems  Center 
Administrative  Management  System  by  including  the  development  of  a  data 
exchange  facility  that  would  enable  the  Systems  Center  to  import  and  export 
data  to  and  from  the  existing  DIA  procurement  and  logistics  systems.  The  CSC 
was  unable  to  migrate  existing  software  applications  to  the  Oracle  data  base  or 
to  develop  a  facility  for  importing  data  because  the  Oracle  commercial  software 
did  not  have  the  import  capabilities  needed  to  support  the  Systems  Center 
acquisition  process.  Additionally,  the  Systems  Center  acquisition  process 
supports  the  Directorates  of  Procurement  and  Logistics  and  must  interface  with 
their  automated  systems.  Oracle  commercial  software  was  not  designed  to 
import  data  from  those  systems;  therefore,  it  could  not  support  other  DIA 
elements.  Consequently,  CSC  was  unable  to  perform  the  tasks  associated  with 
the  import  interface  in  the  delivery  order  30  statement  of  work. 

Future  Direction  of  ARMISS.  In  February  1995,  the  Systems  Center  briefed 
the  new  Deputy  Director,  DIA,  on  the  results  of  the  ARMISS  contract.  As  a 
result  of  that  briefing,  the  Deputy  Director  directed  functional  managers  to 
identify  DoD-standard  systems,  consistent  with  Corporate  Information 
Management  policies  and  principles,  for  satisfying  deficiencies  in  their  existing 
systems  that  ARMISS  had  been  intended  to  remedy.  No  documentation  existed 
to  show  that  the  status  of  the  delivery  orders  was  discussed  at  the  February  1995 
briefing. 

On  September  5,  1995,  the  Chairman  of  the  Configuration  Control  Board  for 
the  Acquisition  System  submitted  a  proposed  tasking  to  COMPEX  Corporation 
to  transition  the  data  in  the  Acquisition  System  to  the  Oracle  data  base  system. 
The  scope  of  the  tasking  included  tasks  to  implement  the  Oracle  products  and  to 
integrate  the  Oracle  financial  software  as  part  of  a  commercial  solution  to  meet 
the  functional  requirements  for  the  Systems  Center  acquisition  process.  The 
Inspector  General,  DIA,  indicated  that  a  planned  follow-on  audit  of  ARMISS 
will  include  the  tasking.  On  October  2,  1995,  the  Director,  DIA,  issued  a 
memorandum  to  the  Systems  Center,  directing  it  to  cease  any  contractual 
funding  for  ARMISS  efforts  specific  to  the  Oracle  products,  pending  completion 
of  our  audit.  The  DIA  has  $1  million  designated  for  ARMISS  in  its  FY  1996 
spending  plan. 
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Summary 


The  DIA  was  unsuccessful  in  achieving  an  integrated  administrative 
management  system  because  of  poor  planning  and  a  lack  of  oversight. 
Specifically,  DIA  committed  itself  to  a  specific  software  solution  without  testing 
to  verify  whether  the  software  would  meet  requirements.  Because  DIA  did  not 
consider  the  limiting  characteristics  of  the  Oracle  commercial  software  and 
because  DIA  did  not  adequately  define  its  requirements  and  business  processes, 
it  selected  software  that  was  unsuited  to  requirements.  Also,  DIA  management 
disregarded  the  required  use  of  the  NSA  finance  and  accounting  system  when 
selecting  a  software  solution  that  was  dependent  on  an  integral  general  ledger 
component.  Further,  DIA  showed  poor  business  judgment  by  buying  large 
quantities  of  software  before  testing  the  adequacy  of  the  software  for  its 
intended  use.  Finally,  DIA  senior  management  approved  the  acquisition  and 
permitted  it  to  proceed,  even  though  serious  problems  became  apparent  within 
2  months  of  contract  award.  These  circumstances  provide  evidence  of  a  need 
for  strengthened  management  controls  over  acquisition  and  contracting  and  a 
formal  reporting  process  for  tracking  the  progress  of  automated  information 
system  projects  in  the  DIA. 

The  Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology 
identified  similar  circumstances  of  poor  planning  and  a  lack  of  oversight  in  a 
Program  Management  Review  that  office  performed  on  the  entire  DIA 
contracting  process.  For  example,  that  review  showed  that  DIA  had  no  overall 
plan  for  fee  acquisition  process  and  feat  requirements  were  ill  defined.  The 
review  also  showed  feat  delivery  orders  should  be  managed  as  stand-alone 
procurements  needing  proper  evaluation,  documentation,  and  management 
oversight  (see  Appendix  B  for  details). 


Recommendations,  Management  Comments,  and  Audit 
Response 


We  recommend  that  the  Director,  Defense  Intelligence  Agency: 

1.  Cease  all  Agency  Resource  Management  Information  Support 
System  development  and  implementation  efforts  until  a  comprehensive 
acquisition  plan  developed  in  accordance  with  Corporate  Information 
Management  policies  and  principles  is  approved. 

Management  Comments.  The  DIA  concurred  wife  fee  recommendation.  The 
Director  stated  feat  fee  DIA  is  pursuing  two  efforts  under  fee  ARMISS, 
development  and  integration  of  a  management  information  system  using  fee 
Oracle  data  base  system  to  satisfy  requirements  of  fee  training  and  personnel 
fimctional  offices.  Further,  all  other  aspects  of  fee  ARMISS  have  ceased,  in 
accordance  wife  fee  direction  of  fee  Director,  DIA,  pending  finalization  of  fee 
audit  report. 
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Audit  Response.  Management  comments  are  responsive.  We  agree  with  the 
decision  to  pursue  efforts  for  the  training  and  personnel  functional  offices  as 
long  as  DIA  does  not  integrate  management  information  systems  developed  for 
those  offices  with  the  Oracle  Government  financial  applications. 

The  DIA  response,  however,  did  not  discuss  specific  development  of  a 
comprehensive  acquisition  plan.  DIA  did  not  provide  the  strategy  for 
implementing  a  corporate  management  information  system,  including  the  Oracle 
Government  Financial  and  Oracle  Personnel  applications.  Specifically,  the 
acquisition  plan  should  cover  the  integration  and  migration  strategy  for 
implementing  all  the  Oracle  COTS  applications  into  the  DIA  architecture  and 
should  identify  the  technical,  business,  management,  and  other  significant 
considerations  or  risks  controlling  or  affecting  the  implementation  strategy. 
Therefore,  we  request  that  the  DIA  provide  additional  comments  on  the 
acquisition  plan  for  the  ARMISS  in  response  to  the  final  report. 

2.  Establish  a  system  of  controls  to  verify  that: 

a.  The  requirements  of  the  Defense  Intelligence  Agency 
intended  users  are  fully  documented  before  an  acquisition  plan  is  approved. 

Management  Comments.  The  DIA  concurred  with  the  recommendation.  The 
comments  state  that  procedures  already  exist  to  ensure  that  documentation  of 
user  requirements  are  incorporated  into  acquisition  plans  and  purchase  requests. 

Audit  Response.  Management  comments  are  not  fully  responsive.  Although 
we  agree  that  the  DIA  has  policies  and  procedures  for  ensuring  that  user 
requirements  are  considered  in  the  acquisition  planning,  those  policies  and 
procedures  were  not  followed  in  acquiring  the  ARMISS.  Although  the  Systems 
Center  sought  involvement  from  DIA-wide  functional  and  organizational  users 
in  developing  ARMISS  requirements  and  in  identifying  automated  technological 
and  functional  process  improvements,  user  involvement  was  not  generally 
effective.  All  interaction  of  the  ARMISS  working  group  and  the  process  action 
team  funnelled  through  the  Systems  Center.  We  found  no  evidence  that  the 
Systems  Center  adequately  considered  or  included  comments  or  concerns 
addressed  by  the  ARMISS  working  group  or  process  action  team  in  the 
ARMISS  acquisition  planning  documents.  Therefore,  in  response  to  the  final 
report,  we  request  that  the  DIA  submit  additional  comments  on  how  control 
verification  will  be  accomplished. 

b.  Procurement  is  limited  to  those  quantities  required  for 
testing  until  the  suitability  of  the  product  or  service  for  the  intended 
purpose  is  demonstrated. 

Management  Comments.  The  DIA  concurred  with  the  recommendation,  but 
provided  no  comments  on  limiting  quantities  until  product  suitability  is 
demonstrated. 

Audit  Response.  DIA  Regulation  65-17  states  that  the  Systems  Center  is 
responsible  to  analyze  user  requirements  using  prototypes  to  die  extent  possible 
to  better  understand  and  refine  requirements  and  to  design  concepts. 
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Nevertheless,  the  ARMISS  request  for  proposal  did  not  require  the  bidding 
vendors  to  demonstrate  the  concept  of  linking  all  seven  administrative  functions 
during  the  pre-award  demonstration. 

Our  evaluations  of  the  vendors'  demonstrations  indicated  that  the  vendors 
demonstrated  only  the  functionality  of  each  application  as  independent  systems. 
Furthermore,  there  was  no  need  to  procure  100  percent  of  the  software  because 
the  DIA  plan,  as  stated  in  the  ARMISS  statement  of  work,  called  for  a  modular 
implementation  approach  of  the  functional  areas  in  order  to  prevent  major 
disruption  to  existing  work  and  to  prevent  downtime  in  critical  functions. 
Commercial  businesses  test  the  features,  performance,  and  capacity  of  COTS 
software  by  buying  a  small  number  of  the  product  for  evaluation,  which  the 
businesses  use  for  prototyping,  development,  and  testing.  If  the  COTS  software 
proves  to  be  suitable,  then  the  deployment  quantities  can  be  purchased  with 
confidence.  We  request  that  in  response  to  the  final  report,  the  DIA  provide 
details  on  limiting  procurement  quantities  until  product  suitability  is  determined. 

c.  Quarterly  status  reports  detailing  cost,  schedule,  and 
performance  data  are  provided  to  senior  Defense  Intelligence  Agency 
managers. 

Management  Comments.  The  DIA  concurred  with  the  recommendation, 
stating  that  senior  management  is  kept  apprised  of  procurement  actions 
appropriately. 

Audit  Response.  Management  comments  are  not  folly  responsive.  The  point 
made  in  our  report  was  that  DIA-wide  systems  like  ARMISS  should  receive  the 
attention  of  the  DIA  Command  Element.  Although  meetings  were  held  to 
discuss  the  status  of  the  ARMISS  effort  at  the  Systems  Center  and  directorate 
levels,  the  concerns  and  issues  that  arose  during  the  preaward  and  postaward 
phases  of  the  ARMISS  contract  required  Command  Element  involvement.  For 
example,  the  System  Center  did  not  keep  senior  managers  at  the  Command 
Element  appropriately  apprised  of: 

o  user  concerns  regarding  the  lack  of  effective  communication  and 
coordination; 

o  the  DIA  Comptroller's  recommendations  to  include  a  plan  for 
reviewing  and  considering  DoD  standard  systems  and  to  procure  the  software  on 
a  module-by-module  basis; 

o  the  effects  that  the  DIA  consolidation  of  accounting  and  finance 
functions  with  the  NS  A  had  on  the  ARMISS  concept  of  operations;  and 

o  the  Chief,  Administrative  and  Resource  Staff  and  Comptroller  concern 
that  the  ARMISS  statement  of  work  did  not  identify  discrete  deliverables 
associated  with  clearly  defined  tasks  that  could  be  individually  costed. 
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We  request  that  in  response  to  the  final  report,  the  DIA  submit  additional 
comments  on  establishing  a  system  of  controls  to  ensure  that  senior  managers 
are  kept  appropriately  apprised  of  important  cost,  schedule,  and  performance 
data  for  important  DIA-wide  acquisition  programs. 

Appendix  D  contains  audit  responses  to  DIA  comments  on  specific  issues  in  the 
finding. 
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Part  II  -  Additional  Information 


Appendix  A.  Scope  and  Methodology 

Scope 


ARMISS  Project.  We  performed  this  audit  jointly  with  the  Inspector  General, 
DIA.  In  response  to  anonymous  allegations  forwarded  by  the  Chairman,  Senate 
Committee  on  Armed  Services  we  reviewed  documentation  relating  to 
two  contracts  for  the  acquisition,  customization,  and  implementation  of 
ARMISS  contract  MDA908-93-C-0027  awarded  June  7,  1993,  to  VSE  for 
$6.9  million;  delivery  order  10,  awarded  September  23,  1993,  for  $568,408; 
and  delivery  order  30,  awarded  September  22,  1994,  for  $578,430  under 
contract  MDA908-93-D-1503. 


Methodology 


At  the  Virginia  Contracting  Activity,  we  examined  VSE  technical  and  cost 
proposals,  statements  of  work  for  the  ARMISS  and  delivery  orders  10  and  30, 
Source  Selection  Evaluation  Board  workbooks  and  files,  source  selection 
evaluation  criteria,  ARMISS  requests  for  information,  VSE  and  progress  and 
status  reports,  and  correspondence  in  the  contract  files  for  ARMISS  and 
delivery  orders  10  and  30. 

At  the  Systems  Center,  we  examined  VSE  deliverables  specific  to  the  functional 
requirements  definition  of  the  Oracle  personnel  and  acquisition  modules  and  to 
studies  and  analyses  regarding  the  transfer  of  DIA  accounting  and  finance 
functions  to  the  NSA  finance  and  accounting  system.  We  also  examined 
documentation  on: 

o  the  acquisition  planning  and  development  and  coordination  of  the 
ARMISS  requirements; 

o  deliverables  specific  to  the  implementation  of  the  Oracle  budget  and 
acquisition  modules; 

o  the  ARMISS  pre-proposal  conference; 

o  the  presolicitation  demonstration  of  the  Oracle  products;  and 

o  correspondence  in  the  contracting  officer  representative  files. 

At  the  DIA  Directorate  of  Administration,  we  examined  the  Administrative 
Functional  Control  Board  records  of  technical  and  functional  issues  regarding 
the  ARMISS.  At  the  office  of  the  DIA  Comptroller,  we  examined  ARMISS 
program  funding  and  information  pertaining  to  the  transition  of  accounting  and 
finance  functions  to  the  NSA.  We  relied  on  Office  of  the  Inspector  General, 
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DoD,  software  engineers  to  perform  a  comparative  analysis  of  the  VSE  and 
CSC  efforts  to  determine  whether  the  delivery  orders  were  duplicative  of  work 
under  the  ARMISS  contract.  The  software  engineers  also  evaluated  the 
technical  implications  of  using  the  commercial  software  applications  to  satisfy 
the  ARMISS  requirements.  All  documentation  examined  was  dated  from  July 
1988  through  September  1995. 

We  interviewed  the  Deputy  Director,  DIA;  the  Director,  National  Military 
Intelligence  Systems  Center  and  members  of  his  staff;  the  former  Director, 
Department  of  Administration;  contracting  officers  at  the  Virginia  Contracting 
Activity;  personnel  at  VSE,  CSC,  BDM  Federal  International  Company,  and 
Oracle  Corporation;  the  Chair  and  members  of  the  Source  Selection  Evaluation 
Board;  program  management  staff  for  ARMISS  and  delivery  orders  10  and  30; 
and  other  DIA  personnel  involved  with  ARMISS  and  delivery  orders  10  and  30. 

Audit  Period,  Standards,  and  Locations.  We  performed  this  economy  and 
efficiency  audit  from  March  through  September  1995  in  accordance  with 
auditing  standards  issued  by  the  Comptroller  General  of  the  United  States,  as 
implemented  by  the  Inspector  General,  DoD,  and,  accordingly,  included  tests  of 
management  controls  considered  necessary.  We  did  not  rely  on  computer- 
processed  data  to  achieve  the  audit  objectives.  Appendix  F  lists  the 
organizations  visited  or  contacted. 


Management  Control  Program 


DoD  Directive  5010.38,  "Internal  Management  Control  Program,"  April  14, 
1987,  requires  DoD  organizations  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are 
operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  Review  of  Management  Control  Program.  We  reviewed 
management  control  procedures  regarding  the  Systems  Center's  program 
management  practices  for  the  acquisition  of  the  ARMISS.  We  also  reviewed 
the  Systems  Center  procedures  pertaining  to  the  use  of  the  DIESCON  for 
ARMISS  implementation  and  integration. 

Adequacy  of  Management  Controls.  We  identified  material  management 
control  weaknesses  for  DIA  as  defined  by  DoD  Directive  5010.38.  DIA 
management  controls  for  acquisition  planning  and  monitoring  of  management 
information  systems  were  not  effective  to  verify  that  user  requirements  and  the 
system  concept  were  fully  analyzed,  documented,  and  coordinated  before  the 
Systems  Center  prepared  die  ARMISS  contract  statement  of  work  and  before  the 
Systems  Center  decided  to  use  commercial  software  as  a  solution  for  satisfying 
the  ARMISS  requirements.  The  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology  Program  Management  Review  (see  Appendix  B) 
identified  similar  management  control  deficiencies.  Also,  controls  were  not 
effective  to  keep  the  DIA  senior  management  regularly  informed  on  the 
progress  of  the  ARMISS  project.  DIA  operating  procedures  permitted  the 
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release  of  the  ARMISS  request  for  proposal  when  acquisition  planning  lacked 
content  and  preparation  and  permitted  the  continuation  of  the  ARMISS  project 
under  the  DIESCON  after  VSE  identified  a  significant  deficiency  in  using  the 
Oracle  commercial  software  to  satisfy  the  ARMISS  concept  design  and 
requirements.  Recommendations  2. a.  and  2.C.,  if  implemented,  will  strengthen 
management  controls  over  the  acquisition  planning  and  the  project  tracking 
processes.  See  Appendix  E  for  a  summary  of  those  benefits.  A  copy  of  the 
report  will  be  provided  to  the  senior  official  responsible  for  management 
controls  in  the  Office  of  the  Assistant  Secretary  of  Defense  (Command, 
Control,  Communications  and  Intelligence). 

Adequacy  of  Management's  Self-Evaluation.  DIA  reviews  management 
controls  every  5  years  and  performs  an  informal  review  every  other  year.  The 
DIA  management  control  program  includes  self-evaluations  by  program 
managers  of  assessable  units.  Program  managers  do  not  use  checklists  for 
evaluating  management  controls.  The  DIA  plans  to  use  checklists  by 
FY  1996  at  the  request  of  the  Inspector  General,  DIA.  The  Inspector  General, 
DIA,  suggested  that  checklists  be  used  as  a  tool  for  assisting  program  managers 
in  preparing  opinions  on  management  controls.  The  checklists  would  be 
provided  to  appropriate  officials  responsible  for  signing  the  annual  statements  of 
assurance.  The  DIA  identified  no  material  weaknesses  in  its  FY  1994  annual 
statement  of  assurance. 
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Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology 


The  Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology 
and  the  Defense  Logistics  Agency  performed  a  Program  Management  Review  of 
the  DIA  contracting  process.  The  review  was  conducted  from  January  23 
through  February  16,  1995.  The  review  team  identified  problems  with  the  DIA 
acquisition  planning  and  requirements  definition  processes.  Regarding 
acquisition  planning,  DIA  had  no  overall  plan  for  the  acquisition  process, 
requirements  were  ill  defined  because  technical  personnel  had  no  incentive  to 
develop  statements  of  work  for  projects  not  likely  to  be  funded,  and  contractor 
responsibilities  were  not  clearly  articulated  in  contractual  instruments.  The 
review  team  concluded  that  the  acquisition  planning  function  had  become  a 
"paper  chase"  within  the  Directorate  of  Procurement  and  that  the  technical 
community  is  not  meeting  the  responsibility  of  writing  statements  of  work. 

Regarding  requirements  definition,  the  review  team  determined  that  DIA  often 
provided  broadly  stated  requirements  in  the  basic  contract  and  made  attempts  to 
narrow  the  scope  through  the  issuance  of  delivery  orders.  The  review  team 
concluded  that  delivery  orders  should  be  managed  as  a  stand-alone  procurement 
with  proper  evaluation,  documentation,  and  management  oversight. 

The  review  team  recommended  that  the  responsibility  for  acquisition  planning 
be  placed  with  staff  knowledgeable  of  the  technical  aspects  and  that  the 
Directorate  of  Procurement  be  taken  out  of  the  business  of  monitoring 
acquisition  planning  actions  of  $100,000  or  more.  The  review  team  also 
recommended  that  acquisition  planning  concepts  in  DIA  Manual  44-2, 
"Acquisition,"  be  revised  to  include  the  development  of  meaningful  planning 
tools  to  relate  an  acquisition  strategy  to  a  particular  requirement.  Further,  the 
review  team  recommended  ensuring  that  statements  of  work  received  from  the 
requiring  activities  are  well-defined  in  order  to  promote  adequate  competition 
and  that  contracting  officers  and  technical  analysts  have  a  comprehensive 
understanding  of  the  statement  of  work  in  order  to  validate  technical  proposals. 
The  DIA  is  in  the  process  of  coming  up  with  an  action  plan  to  implement  each 
recommendation. 


Inspector  General,  DoD 


Inspector  General,  DoD,  Report  No.  92-084,  "Acquisition  of  Automated  Data 
Processing  Equipment  By  die  Defense  Intelligence  Agency,"  May  1,  1992, 
identifies  a  lack  of  formal  acquisition  planning.  Specifically,  the  audit  found  no 
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evidence  that  DIA  employed  a  structured,  disciplined  acquisition  planning 
process;  that  planning  documents  specified  the  acquisition  approaches 
considered  or  rationales  used  by  DIA  in  making  key  acquisition  decisions;  or 
that  DIA  adequately  coordinated  the  acquisition  objectives,  schedules, 
problems,  status,  and  plans.  In  addition,  inadequate  planning  directly 
contributed  to  questionable  DIA  actions  and  decisions  regarding  contract 
coordination,  type,  and  costing  methodology. 

The  report  recommends  revising  DIA  Manual  44-2  to  indicate  the  considerations 
and  requirements  in  planning  acquisitions  totaling  more  than  $15  million  in  any 
fiscal  year.  DIA  Manual  44-2  was  revised  to  include  a  list  of  planning  actions 
for  verifying  that  requiring  activity  acquisition  plans  are  completed.  The  report 
also  recommends  developing  and  documenting  procedures  to  ensure  that  future 
requirements-type  contracts  are  based  on  and  developed  from  well-defined  needs 
of  the  users.  DIA  Manual  44-2  was  also  revised  to  require  supporting 
documentation  from  the  users  for  all  indefinite-delivery-type  contracts. 


Inspector  General,  DIA 


Inspector  General,  DIA,  Project  No.  92-1215-0A-001,  "Management  and 
Control  of  Commercial-off-the-Shelf  Software,"  August  22,  1994.  The  report 
identifies  inconsistent  and  unreliable  processes  for  meeting  user  requirements. 
Specifically,  the  justification  and  documentation  for  commercial  software 
requests  were  either  nonexistent  or  incomplete,  a  process  for  review  and 
approval  of  requests  was  not  established,  and  die  processing  of  subsequent 
software  acquisition  actions  was  inconsistent. 

The  report  recommends  establishing  guidance  for  the  review  of  all  software 
requests,  implementing  procedures  to  ensure  all  actions  for  software  requests  are 
fully  documented,  and  implementing  a  method  to  track  user  requests. 

The  DIA  concurred  with  all  recommendations  and  has  either  completed  or  is  in 
the  process  of  planning  action  for  each  recommendation. 

Inspector  General,  DIA,  Project  No.  94-1545-0A-014,  "Automated  Data 
Processing  Hardware  Inventory,"  August  2,  1995.  The  report  states  that 
automated  data  processing  inventory  records  contained  errors  and  that  the 
accumulation  of  automated  data  processing  inventory  in  the  DIA  warehouse  was 
primarily  caused  by  inadequate  planning.  The  Inspector  General,  DIA, 
recommended  implementing  recommendations  made  by  the  Business  Process 
Improvement  Team  V  Acquisition  Baseline  Workshop  to  improve  procurement 
tracking.  The  DIA  has  not  completed  a  formal  action  plan;  however,  corrective 
actions  are  in  progress  for  all  recommendations. 
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Allegation  1.  In  advance  of  issuing  the  competitive  solicitation,  the  ARMISS 
program  manager  preselected  an  Oracle  Corporation  product,  "Oracle 
Financial. " 

Audit  Result.  We  were  unable  to  substantiate  the  allegation.  The  ARMISS 
program  manager  and  other  DIA  personnel  attended  a  demonstration  of  the 
Oracle  financial  products  before  the  issuance  of  a  competitive  solicitation. 
However,  the  solicitation  did  not  specify  that  the  vendors  propose  Oracle 
products,  in  whole  or  in  part,  to  satisfy  the  ARMISS  request  for  proposal.  The 
vendors  could  propose  any  commercial  software  product  that  offered  an 
integrated  solution  for  satisfying  the  ARMISS  requirements. 

Additionally,  the  seven  members  on  the  ARMISS  Source  Selection  Evaluation 
Board  independently  read  and  evaluated  the  ARMISS  cost  and  technical 
proposals,  documenting  their  scores  in  individual  evaluation  workbooks.  In 
addition  to  the  seven  members,  nine  functional  advisors  independently  evaluated 
the  proposals.  The  proposals  were  evaluated  on  the  mechanics  of  the  proposals 
and  the  vendors'  proposed  solutions  rather  than  the  software  products  proposed 
by  the  vendors.  Once  the  individual  scoring  was  completed,  the  Source 
Selection  Evaluation  Board  met  to  come  to  an  overall  consensus  score. 

DIA  received  three  proposals  for  integrating  its  administrative  functions. 
One  vendor  was  disqualified  because  it  proposed  a  software  development 
solution  rather  than  a  commercial  software  solution.  The  remaining 
two  vendors,  VSE  and  Integrated  Microcomputer  Systems  each  proposed  Oracle 
software  products  and  Banner  commercial  training  software.  In  addition,  VSE 
proposed  STARBUCS  travel,  and  Integrated  Microcomputer  Systems  proposed 
a  Government-off-the-shelf  travel  program  to  satisfy  the  ARMISS  request  for 
proposal.  We  reviewed  the  source  selection  process  and  found  it  to  be  fair  and 
reasonable. 

Allegation  2.  DoD  acquisition  regulations  were  not  followed.  Specifically,  no 
formal  funding  plan,  no  formal  acquisition  plan,  and  no  requirements  definition 
were  completed  for  the  ARMISS  project. 

Audit  Results.  The  allegation  was  substantiated.  The  DIA  did  not  follow 
policies  and  procedures  in  DoD  Directive  8120.1,  specific  to  life-cycle 
management  review  and  milestone  approval  for  automated  information  systems 
projects.  The  DIA  did  not  have  a  formal  funding  plan  for  ARMISS.  In 
addition,  senior  management  oversight  permitted  the  realignment  of  funds  from 
other  Systems  Center  budgets  to  fund  the  ARMISS  contract  and  contract 
modifications.  The  Systems  Center  had  a  formal  acquisition  plan;  however,  it 
was  not  sufficient.  Specifically,  the  Systems  Center  did  not  adequately  define 
the  requirements  for  ARMISS,  adequately  analyze  and  explore  the  ARMISS 
concept  design  before  procuring  100  percent  of  the  assumed  quantities  of 
software,  or  consider  the  technical  implications  associated  with  the  requirement 
for  DIA  to  use  the  NSA  accounting  and  finance  system.  See  the  Finding  for 
details. 
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Allegation  3.  The  ARMISS  Program  Manager  applied  undue  pressure  on  the 
ARMISS  evaluation  committee  members  and  contracting  staff  to  award  the 
ARMISS  contract. 

Audit  Result.  We  partially  substantiated  the  allegation.  We  interviewed  three 
of  the  seven  ARMISS  Source  Selection  Evaluation  Board  members,  and  two  of 
them  stated  that  they  felt  pressure  by  the  ARMISS  program  manager  to  quickly 
award  the  contract.  However,  the  two  members  asserted  that  outside  pressure 
did  not  influence  their  decision.  We  also  interviewed  the  contracting  officer 
who  awarded  the  ARMISS  contract.  He  suggested  recompeting  the  ARMISS 
request  for  proposal  because,  although  the  bids  were  acceptable,  they  were  not 
exceptional.  Also,  the  contracting  officer  suggested  revamping  the  ARMISS 
statement  of  work  because  the  contractor  tasks  were  vague.  However,  we 
determined  that  the  ARMISS  program  manager  applied  pressure  to  award  the 
contract  quickly. 

Allegation  4.  The  ARMISS  contract  was  permitted  to  expire  because: 

o  the  Oracle  products  could  not  be  modified  to  meet  DIA  needs,  and 

o  the  contractor  could  not  perform  because  of  DIA  undefined  functional 
requirements. 

Audit  Results.  The  allegation  was  substantiated.  The  Oracle  commercial 
software  modules  did  not  readily  conform  to  the  DIA  business  practices  due  to 
regulatory  requirements,  to  the  lack  of  functionality  essential  to  business 
processes,  or  to  design  features  inherent  in  the  Oracle  commercial  software. 
For  the  Oracle  commercial  software  to  meet  DIA  needs,  the  software  required 
extensive  modification  to  the  data  base  table  structures  and  development  of  an 
import  facility.  Oracle  Corporation  considers  changes  of  this  magnitude  to  be 
risky  which  could  result  in  relinquishing  the  software  warranty  and  maintenance 
support  agreements.  See  the  Finding  for  details. 

Allegation  5.  The  ARMISS  program  manager  spent  more  than  $1  million  in  an 
attempt  to  use  the  Oracle  commercial  software  to  satisfy  resource  management 
deficiencies  in  the  Systems  Center  operational  systems. 

Audit  Results.  The  allegation  was  substantiated.  The  ARMISS  program 
manager  spent  about  $1.1  million  under  the  DIESCON  to  implement  the  Oracle 
financial  modules.  The  work  required  under  DIESCON  delivery  orders  10  and 
30  were  to  provide  the  Systems  Center  with  a  fully  functional,  integrated  system 
for  tracking  and  monitoring  budget  expenditures  and  automated  data  processing 
acquisitions.  However,  die  original  intent  of  using  the  DIESCON  was  to 
integrate  the  customized  software  into  the  DIA  automated  environment  for 
satisfying  the  DIA-wide  ARMISS  requirement.  Further,  the  scope  of  work 
under  delivery  order  30  included  replacing  the  Systems  Center  Acquisition 
System,  which  was  an  existing,  operational  system.  However,  CSC  could  not 
replace  the  Acquisition  System  due  to  the  risks  associated  with  importing  data 
from  external  procurement  and  logistics  systems.  See  the  Finding  for  details. 
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Allegation  6.  The  ARMISS  program  manager  redirected  FY  1994  funds  to 
acquire  more  than  $1  million  dollars  in  hardware  to  support  the  delivery  order 
efforts  under  DIESCON. 

Audit  Results.  We  were  unable  to  substantiate  the  allegation.  DIA  did  not 
have  an  adequate  mechanism  for  tracking  the  purchase  of  automated  data 
processing  equipment  to  specific  programs.  DIA  purchases  automated  data 
processing  hardware  under  its  Systems  Acquisition  and  Support  Services 
hardware  contract.  Purchase  requests  may  include  requirements  for  more  than 
one  project,  and  the  requirements  were  not  well  defined.  Therefore,  DIA  had 
no  assured  method  of  accounting  for  all  hardware  purchased  under  the 
DIESCON  efforts.  To  substantiate  the  allegation,  we  would  have  had  to  match 
purchase  requests  for  the  procurement  of  all  hardware  in  FY  1994  against  all 
invoices  and  would  have  had  to  perform  a  physical  inventory  of  the  hardware 
shipped.  Inspector  General,  DIA,  reports  on  "Management  and  Control  of 
Commercial-off-the-Shelf  Software"  and  on  "Automated  Data  Processing 
Hardware  Inventory,"  identified  similar  problems  in  tracking  purchases  to 
particular  projects  (see  Appendix  B  for  details). 

Allegation  7.  Attempts  have  been  made  to  cover  the  issue  or  hide  the  fact  that 
the  Oracle  software  does  not  address  DIA  management  information  needs. 

Audit  Results.  The  allegation  was  not  substantiated.  We  found  that  the 
Systems  Center  failed  to  keep  DIA  senior  management  regularly  informed  of 
the  attempts  that  VSE  or  CSC  made  on  customizing  the  Oracle  software  or  on 
implementing  the  software  into  the  DIA  automated  environment.  Further,  the 
Systems  Center  failed  to  make  DIA  senior  management  aware  of  plans  to  use 
the  DIESCON  as  a  vehicle  to  continue  implementation  of  the  Oracle 
commercial  software.  However,  there  was  no  evidence  of  an  overt  or  conscious 
action  to  hide  the  difficulty  in  customizing  the  Oracle  software  or  use  of  the 
DIESCON  from  DIA  senior  management.  See  the  Finding  for  details. 

Allegation  8.  There  has  been  virtually  no  real  agency  oversight  of  the  ARMISS 
project. 

Audit  Results.  The  allegation  was  substantiated.  Before  the  award  of  the 
ARMISS  contract,  the  DIA  command  element  formally  met  with  ARMISS 
program  management  to  discuss  the  broad  objectives  of  the  acquisition  of 
ARMISS.  Those  meetings  were  held  August  3,  1992,  and  March  3,  1993. 
After  contract  award  in  June  1993  through  October  1994,  the  Systems  Center 
briefed  DIA  senior  management  twice  on  the  status  of  the  ARMISS  project, 
despite  the  deficiencies  identified  in  the  requirements  documentation  and  in  the 
Oracle  software  product  to  satisfy  the  system  concept  design.  Finally,  the 
Systems  Center  did  not  brief  DIA  senior  management  until  March  1995  on  the 
two  delivery  orders  initiated  in  September  1993  and  1994.  See  the  Finding  for 
details. 
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Management  Comments 


The  DIA  provided  detailed  comments  on  the  finding  in  the  draft  report.  Those 
management  comments  and  our  audit  responses  follow. 


DIA  Comments 


Comments.  The  draft  audit  report  overlooks  the  strategy  regarding  the  use  of 
COTS  products  in  implementing  the  ARMISS  concept. 

Audit  Response.  We  disagree.  The  intended  use  of  a  COTS  solution  does  not 
mitigate  risks  of  system  development  efforts.  Implementation  of  COTS 
products  requires  the  same  development  strategies  as  software  development 
efforts  and  imposes  the  same  risks,  life-cycle,  development,  implementation, 
deployment,  maintenance  and  other  concerns  of  any  major  software 
development  effort.  In  accordance  with  the  Air  Force  Software  Technology 
Support  Center,  "Guidelines  for  Successful  Acquisition  and  Management  of 
Software  Intensive  Systems,"  February  1995,  use  of  COTS  products  does  not 
prevent  agencies  from  using  traditional  prototyping  procedures.  In  fact,  the 
Air  Force  guidance  states  "...  it  is  often  difficult  to  integrate  all  the  COTS 
applications  needed  to  provide  the  required  functionality."  To  alleviate  this 
burden,  traditional  approaches  must  be  tailored  to  accommodate  the  acquisition 
and  management  of  COTS  solutions  and  to  reduce  risk. 

One  way  to  reduce  risk  is  to  allow  the  demonstration  phase  of  the  COTS 
product  to  reach  the  level  of  maturity  where  prototyping  integrates  the  selected 
products  in  vertical  and  horizontal  dimensions,  creating  a  fairly  complete  system 
implementation.  This  phase  can  also  significantly  reduce  the  risk  in  the 
requirements  baseline  and  system  design,  including  the  COTS  product  selection. 
Therefore,  it  is  important  not  to  procure  100  percent  of  the  assumed  quantities 
in  case  significant  risk  is  identified. 

The  intent  of  our  report  was  not  to  imply  that  COTS  was  a  bad  idea.  We 
clearly  understood  that  the  COTS  products  would  not  satisfy  100  percent  of  the 
ARMISS  requirements  and  that  extensions  to  data  fields  or  "flexfields"  would 
be  used  to  effect  the  necessary  customization  rather  than  modifying  source 
codes.  However,  because  the  ARMISS  contract  with  VSE  was  the  first  effort  to 
apply  a  bundled  suite  of  COTS  products  to  integrate  seven  administrative 
functions,  the  DIA  should  have  tailored  the  effort  by  using  software 
development  standards  and  practices  as  a  guideline. 

Comments.  "Proof  of  concept"  had  been  substantiated  by  the  responses  to  the 
previous  Government  issued  Request  for  Information,  which  demonstrated  that 
COTS  products  indeed  did  exist  in  the  marketplace. 
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Audit  Response.  We  disagree.  The  request  for  information  identifies 
availability  of  sources  capable  of  satisfying  the  Government's  requirements; 
however,  the  request  for  information  does  not  prove  that  the  concept  works  or 
that  the  product  can  be  successfully  implemented.  Relying  solely  on  product 
documentation  or  demonstration  is  an  invitation  to  failure,  according  to  the  Air 
Force  guidelines.  DIA  should  have  investigated  other  sites  similar  to  the  DIA 
structure  where  the  complete  suite  of  COTS  products  had  been  implemented. 

Further,  given  the  imposed  legal  and  regulatory  constraints  on  certain  DIA 
business  practices,  that  is,  budget  and  personnel,  the  ARMISS  concept  of 
operations  was  not  viable.  Adequate  demonstrations  of  the  product,  to  include 
integrating  the  selected  products  in  vertical  and  horizontal  dimensions,  mitigates 
risk  in  die  requirements  baseline  and  creates  a  fairly  complete  system 
implementation. 

Comments.  The  DIA  nonconcurs  with  the  statement  that  allegations  can  be 
substantiated. 

Audit  Response.  We  disagree.  The  documentary  and  testimonial  evidence 
obtained  during  the  audit  was  sufficient  to  substantiate  or  partially  substantiate 
six  of  the  allegations. 

Comments.  Given  the  DIA  acceptance  of  the  VSE  proposed  schedule  for 
implementation  within  2  years,  the  plan  to  get  users  involved  in  the  products, 
and  the  good  price  being  offered,  the  DIA  agreed  to  buy  100  percent  of  the 
assumed  quantities  as  a  good  business  decision.  Due  only  to  later  events,  when 
the  ARMISS  implementation  plan  radically  changed,  the  DIA  leadership 
stopped  the  financial  management  module  indefinitely,  and  implementation  went 
into  hiatus,  did  the  number  of  licenses  seem  an  error.  More  than  200  of  the 
500  licenses  procured  under  the  ARMISS  contract  are  being  used. 

Audit  Response.  We  disagree  with  the  DIA  decision  to  procure  100  percent  of 
the  assumed  quantities.  In  November  1991,  the  DIA  Comptroller  stated  that  a 
module-by-module  approach  should  be  adopted  and  that  the  contract  should  be 
constructed  so  as  to  provide  maximum  flexibility  to  the  DIA  to  continue, 
exclude,  or  discontinue  some  module  efforts. 

For  clarification,  the  licenses  currently  being  used  are  only  portions  of  the 
procured  software,  that  is,  Oracle  data  base  system.  Banner  Student  System,  and 
portions  of  the  Oracle  financial  module. 

Comments.  Both  of  the  two  viable  contractors  bidding  on  the  contract 
demonstrated  the  proof  of  concept  with  the  Oracle  products  and  determined  it 
was  feasible.  The  issue  was  that  leadership  was  not  willing  to  accept  a  less  than 
100-percent  solution  and  change  business  processes  to  maximize  the  usefulness 
of  the  product  applications. 

Audit  Response.  In  addition  to  responses  already  provided,  legal  and 
regulatory  constraints  as  well  as  lack  of  functionality  prevented  the  DIA  from 
using  the  Oracle  COTS  applications  as  intended.  Specifically,  the  Oracle 
personnel  module  was  not  a  Government  application  and  did  not  contain  forms 
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or  critical  data  fields  needed  to  process  mandatory,  Government-specific 
personnel  actions;  Oracle  budget  module  duplicated  functionality  inherent  in  the 
NSA  accounting  and  finance  system,  the  use  of  which  was  directed  by  the 
Under  Secretary  of  Defense  (Comptroller);  the  Oracle  acquisition  module  did 
not  contain  key  functionality  such  as  a  function  that  supports  the  processes  of 
receiving,  configuring,  installing,  and  maintaining  hardware;  and  Oracle 
Corporation  did  not  have  a  standard  interface  to  import  data  from  external 
systems  in  the  DIA  environment. 

Comments.  The  fact  that  the  NSA  replaced  the  Air  Force  does  not  alter  the 
fact  that  a  data  feed  to  an  outside  financial  and  accounting  function  was 
considered  and  deemed  feasible.  Further,  no  one  had  proposed  or  considered, 
at  any  time  during  the  planning  or  acquisition  phase,  that  the  Oracle  financial 
management  product  would  perform  as  the  DIA  system. 

Audit  Response.  We  disagree.  As  pointed  out  in  the  report,  the  decision  to 
transfer  accounting  and  finance  functions  to  the  NSA  was  made  in  August  1992. 
Even  if  no  one  fully  understood  that  the  NSA  accounting  system  would  perform 
as  the  DIA  system,  the  Systems  Center  should  have  included  the  NSA 
accounting  system  in  the  ARMISS  statement  of  work  as  a  system  that  would 
need  to  interface  with  ARMISS.  Furthermore,  DIA  officials  proposed  or 
considered  the  transition  to  the  NSA  accounting  system  during  the  planning 
stage.  For  example,  on  February  23,  1993,  the  Office  of  DIA  Comptroller 
notified  the  Systems  Center  that  DIA  and  NSA  will  use  the  standard  DoD  pay 
and  finance  and  accounting  systems.  The  Systems  Center,  as  the  DIA  data 
administrator,  could  not  prescribe  a  system  solution  for  pay  or  finance  and 
accounting  systems,  no  matter  which  standards  the  DoD  directed  to  be  used. 
The  DIA  would  have  to  develop  the  interfaces  needed  between  ARMISS  and  the 
standard  systems  implemented  at  NSA. 

In  September  1992,  another  individual,  a  user  on  the  process  action  team, 
addressed  potential  implications  of  using  the  NSA  accounting  system.  He  stated 
that: 


...  it  is  my  understanding  that  the  DIA  Comptroller  has  been 
directed  by  the  Defense  Finance  and  Accounting  Service  to  use  the 
finance  package  of  the  NSA.  ...  As  you  probably  know,  all  of  the 
NSA's  modules  which  DIA  wishes  to  have  are  already  up  and 
running.  If  DIA  went  out  onto  the  streets  and  bought  another 
ARMISS  package,  how  complete  would  the  DIA  ARMISS  modules 
like  logistics,  acquisition,  travel  and  budget  be  if  the  NSA  based 
finance  package  is  not  compatible? 

Comments.  The  contract  was  not  terminated  because  the  external  interfaces 
were  too  difficult  or  the  Oracle  COTS  products  were  not  suited  to  satisfying  the 
ARMISS  goals. 

Audit  Responses.  We  disagree  and  stand  by  our  position  as  stated  in  the 
report.  The  implementation  strategy  was  significantly  affected  because  of  the 
Government's  failure  to  provide  the  documented  business  processes  to  VSE,  the 
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unanticipated  difficulty  in  customizing  the  software,  and  the  conflict  between 
the  ARMISS  concept  of  operations  and  the  use  of  the  NSA  accounting  and 
finance  system. 

Comments.  The  statements  and  conclusions  in  the  report  regarding  the  Chief 
(as  the  ARMISS  program  manager)  of  the  Logistics  Division's  professional 
conduct  with  VSE,  die  DIESCON  delivery  orders,  and  senior  management 
involvement,  are  inaccurate. 

Audit  Response.  The  ARMISS  program  manager  circumvented  procurement 
practices  on  a  number  of  occasions  during  the  ARMISS  effort  as  noted  in  the 
report.  The  ARMISS  program  manager  bypassed  the  Contracting  Officer 
Technical  Representative  as  well  as  VSE,  the  prime  contractor,  in  executing  the 
ARMISS  contract.  The  ARMISS  program  manager  called  an  emergency 
Administrative  Functional  Control  Board  meeting  in  July  1993  to  inform 
functional  users  that  they  needed  to  answer  some  questions,  at  the  request  of 
VSE,  as  quickly  as  possible  to  facilitate  the  interview  process.  VSE  informed 
the  Agency  Business  Manager,  Co-Chair  of  the  Board,  that  the  questions  and 
answers  were  useless  and  that  the  ARMISS  program  manager  and  Oracle 
Corporation  had  arranged  for  the  question  and  answer  drill  without  VSE 
knowledge. 

On  August  31,  1993,  VSE  held  an  ARMISS  meeting  with  DIA  officials.  At  the 
conclusion  of  that  meeting,  the  President  and  Chief  Executive  Officer,  VSE, 
spoke  to  the  Director  of  the  DIA  Contracting  Activity,  regarding  the 
coordination  process  and  flow  of  information  between  the  ARMISS  program 
manager  and  the  ARMISS  Contracting  Officer  Technical  Representative.  The 
President  stated  that  the  Contracting  Officer  Technical  Representative  was  not 
kept  informed  of  tasking  assignments  given  by  the  ARMISS  program  manager 
to  the  VSE  team  and  of  meetings  that  he  held  with  VSE  or  Oracle  employees. 

Comments.  The  DIA  nonconcurs  with  the  statement  in  the  report  that, 
"...  the  ARMISS  concept  continued  to  be  invalid  because  the  Oracle  budget 
module  needed  to  interface  with  the  NSA  system  to  provide  an  integrated 
solution. "  The  suite  of  Oracle  COTS  products  can  work  either  independently  or 
in  conjunction  with  each  other.  Also,  implementation  of  the  financial  module 
ceased  because  of  policy  decisions  made  by  senior  management  regarding  the 
use  of  the  NSA  system,  not  because  the  ARMISS  concept  was  not  feasible. 

Audit  Response.  The  fact  that  the  Oracle  COTS  products  could  not  be 
physically  linked  to  the  NSA  accounting  and  finance  system  was  the  reason  that 
the  ARMISS  concept,  as  envisioned,  was  no  longer  viable.  It  is  true  that  the 
DIA  Comptroller  did  not  object  to  the  System  Center's  initiative  to  install  and 
integrate  the  Oracle  financial  module  into  the  Systems  Center's  environment  to 
replace  its  WANG  system  and  to  serve  as  a  resource  manager's  tool.  However, 
the  Comptroller,  as  well  as  other  cognizant  officials  within  the  DIA 
Comptroller's  office  stated  that  the  Systems  Center  had  no  valid  need  for  a 
general  ledger  and  that  the  Oracle  financial  module  was  not  an  adequate 
replacement  for  the  WANG  system  because  much  of  the  capability  inherent  in 
the  Oracle  product  could  not  be  used  due  to  the  restrictions  imposed  by  the  DIA 
Comptroller. 
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The  DIA  Comptroller  indicated  that  there  could  be  no  operational  processing  in 
ARMISS  and  no  arrangements  that  constituted  a  financial  commitment.  Use  of 
the  Oracle  financial  module  as  a  resource  manager’s  tool  was  limited  to  an 
independent  system  because  the  module  could  not  link  to  the  NSA  accounting 
and  finance  system.  The  Oracle  financial  module  could  be  used  as  an  interface 
tool  to  provide  access  to  the  raw  data.  However,  the  concept  of  a  centralized, 
integrated  client-server  environment  was  no  longer  viable. 

Comments.  The  DIA  nonconcurs  with  the  rationale  regarding  Allegation  2. 
Specifically,  no  formal  funding  plan,  no  formal  acquisition  plan,  and  no  formal 
requirements  definition  existed  for  the  ARMISS  project. 

Audit  Response.  The  DIA  had  a  formal  acquisition  plan  signed  by  all 
appropriate  individuals  that  met  the  documentation  requirements  established  by 
the  DIA  Contracting  Activity.  However,  the  content  of  the  acquisition  plan  was 
not  sufficient,  as  stated  in  our  report. 

Comments.  The  DIA  nonconcurs  with  the  rationale  regarding  the  cited 
substantiation  for  Allegation  3. 

Audit  Response.  We  obtained  testimonial  evidence  from  two  Source  Selection 
Evaluation  Board  members  as  well  as  the  ARMISS  Contracting  Officer  that  they 
felt  pressure  from  the  ARMISS  program  manager  to  award  the  ARMISS 
contract  quickly.  The  urgency  to  award  the  ARMISS  contract  in  June  1993 
represented  poor  planning  because  during  the  first  few  months  of  the  contract, 
VSE  was  directed  to  implement  the  finance  and  acquisition  module.  Fourth- 
quarter  work  loads  for  those  two  offices,  however,  forced  significant  changes  to 
the  ARMISS  implementation  schedule,  and  we  believe  there  was  no  need  to 
award  the  contract  quickly. 

Comments.  The  DIA  nonconcurs  with  the  rationale  regarding  the 
substantiation  for  Allegation  4. 

Audit  Response.  The  DIA  was  not  able  to  implement  the  Oracle  personnel 
module  because  the  VSE  analysis  of  the  software  identified  that  it  could  not  be 
readily  tailored  within  the  design  of  the  COTS  product  capabilities  and  would 
require  extensive  modification  to  source  code,  which  would  jeopardize 
warranties  and  maintenance  support  agreements.  The  personnel  functional  area 
needed  a  solution  that  would  accommodate  the  Government-mandated  legal  and 
regulatory  business  practices.  Furthermore,  the  VSE  analysis  of  the  Oracle 
acquisition  module  identified  a.  lack  of  functionality  and  risk  associated  with 
importing  data  from  existing  procurement  and  logistics  systems  to  the 
acquisition  module.  VSE  indicated  that  a  concept  of  operations  needed  to  be 
developed  to  determine  how  the  existing  Oracle  systems  would  interface. 

Comments.  The  DIA  nonconcurs  with  the  rational  regarding  the  substantiation 
for  Allegation  5. 

Audit  Response.  As  a  result  of  technical  complexities  associated  with  the 
interface  development  and  the  integral  general  ledger,  the  DIA  should  not  have 
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continued  efforts  to  upgrade  or  replace  its  existing  administrative,  management 
information  systems  under  the  DIESCON  contract  until  the  Systems  Directorate 
developed  and  evaluated  a  detailed  implementation  strategy. 

During  the  audit,  we  were  made  aware  that  the  Logistics  Division  had  plans  to 
connect  the  general  ledger  to  the  local  area  network  and  to  eventually  link  the 
purchase  module  to  the  general  ledger  so  that  the  Logistics  Division  could  build 
purchase  requests  in-house.  According  to  Logistics  Division  personnel,  they 
were  planning  to  define  further  CSC  support  using  technical  expertise  of 
COMPEX  Corporation  personnel.  The  support  consisted  of  transitioning  the 
data  in  the  Acquisition  System  to  the  Oracle  data  base  system  and  transitioning 
existing  user  applications  to  the  Oracle  Government  Financial  modules. 

Comments.  The  DIA  nonconcurs  with  the  rationale  regarding  the 
substantiation  for  Allegation  8. 

Audit  Response.  We  found  no  evidence  that  users  were  expecting  the  COTS 
products  to  provide  a  100-percent  solution.  Rather,  users  were  merely  looking 
for  a  solution  to  satisfy  their  functional  requirements,  to  include  replacing 
antiquated  systems.  When  users  learned  that  the  Oracle  COTS  products  did  not 
readily  conform  to  significant  business  practices,  lacked  essential  functionality, 
or  lacked  a  standard  interface  to  external  systems,  users  began  looking  at  the 
upcoming  (February  1995)  standard  DoD-wide  corporate  information 
management  systems  as  a  solution,  in  accordance  with  the  direction  of  the 
Deputy  Director,  DIA. 
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Resulting  From  Audit 

Recommendation  Amount  and 

Reference  Description  of  Benefit  Type  of  Benefit 

1.  Economy  and  Efficiency.  Stops  Nonmonetary, 

further  acquisition  effort  until 
planning  documentation  is 
developed  in  compliance  with 
existing  DoD  guidance. 

2. a.  Management  Controls.  DIA  will  Nonmonetary, 

include  the  users  in  development  of 
acquisition  plans. 

2.b.  Economy  and  Efficiency.  DIA  Undeterminable.* 

resources  for  commercial  software 
acquisitions  will  be  spent  on 
technically  feasible  and  cost- 
effective  alternatives. 

2.c.  Management  Controls.  Establishes  Nonmonetary, 

procedures  to  track,  monitor,  and 
report  the  status  of  automated 
information  system  projects  on  a 
regular  basis. 


♦Monetary  benefits  were  undeterminable  because  the  DIA  resources  earmarked 
for  commercial  software  acquisitions  may  be  used  for  bulk  purchases  without 
testing  to  reduce  the  risk  associated  with  integration  into  system  designs. 
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Appendix  F.  Organizations  Visited  or  Contacted 

Office  of  the  Secretary  of  Defense 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence), 
Washington,  DC 


Other  Defense  Organizations 

Defense  Intelligence  Agency,  Washington,  DC 
National  Security  Agency,  Fort  George  G.  Meade,  MD 


Non-Government  Organizations 

Value  Systems  Engineering  Corporation,  Alexandria,  VA 
Computer  Sciences  Corporation,  Falls  Church,  VA 
Oracle  Corporation,  Bethesda,  MD 
BDM  Federal  International  Company,  McLean,  VA 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 

Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence) 
Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Intelligence  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
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Non-Defense  Federal  Organizations  and  Individuals 

Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 

Senate  Committee  on  Armed  Services 

Senate  Committee  on  Governmental  Affairs 

Senate  Select  Committee  on  Intelligence 

House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 
House  Permanent  Select  Committee  on  Intelligence 
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Defense  Intelligence  Agency  Comments 


DEFENSE  INTELLIGENCE  AGENCY 


WASHINGTON.  D.C.  20340- 


U-003/S-04 


W3W  7996 


MEMORANDUM  FOR  THE  DIRECTOR,  READINESS  AND  OPERATIONAL  SUPPORT  DIRECTORATE 
ATTENTION:  Audit  Program  Director 

SUBJECT:  Audit  Report  on  Agency  Resource  Management  Information  Support 

System  (Project  No.  5RF-5025) 

Reference:  DoD  16  memorandum  of  31  October  1995,  subject  as  above 


1.  The  Defense  Intelligence  Agency  (DIA)  concurs  with  the  recommendations 
proposed  by  the  draft  audit  report  on  the  Agency  Resource  Management 
Information  Support  System  (ARMISS) .  Regarding  the  report's  recommendations, 
the  following  actions  are  already  underway  or  are  in  place  within  the  Agency: 


a.  The  Director,  DIA,  directed  that  those  aspects  of  ARMISS  relating  to 
Training  (BANNER)  and  Personnel  continue  to  be  pursued,  but  all  other  aspects 
of  ARMISS  are  being  held  in  abeyance  pending  finalization  of  the  audit  report. 
At  that  time,  DIA  will  review  the  corporate  management  information  system  and 
determine  further  actions.  The  problems  and  needs  upon  which  the  requirement 
for  a  corporate  management  Information  system  have  been  long  based  still 
exist. 

b.  Procedures  are  in  place  within  DIA  to  ensure  requirements  reflect  user 
needs,  that  multiple  authorities  are  involved  in  the  coordination  and  approval 
of  acquisition  actions  to  ensure  the  Agency’s  best  interests,  and  that  senior 
management  is  kept  apprised  of  procurement  actions  appropriately. 

2.  In  order  to  ensure  as  accurate  and  true  a  record  is  presented,  DIA  is 
providing  comments  regarding  the  draft  report  statements,  conclusions  and 
findings.  DIA  believes  that  none  of  the  allegations  can  be  substantiated. 

The  enclosed  comments  and  change  requests  are  provided  to  detail  DIA’s 
concerns.  They  are  arranged  in  three  sections:  Introduction,  General /Overall 
Comments;  and  Detailed  Specifics  In  order  of  the  report. 

3.  Request  the  draft  audit  report  be  modified  pursuant  to  the  enclosed. 


FOR  THE  DIRECTOR 


1.  Enclosure  a/s 


_ 

RICHARD  B.  WALKER 
Director,  National  Military 
Intelligence  Systems  Center 
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DEFENSE. INTELLIGENCE  AGENCY  COMMENTS  ON 
DoD  IG  DRAFT  AUDIT  REPORT  ON  ARMISS  (PROJECT  NO.  5RF-5025) 


INTRODUCTION 


The  Defense  Intelligence  Agency  non-concurs  with  the  draft  audit  report  on 
the  Agency  Resource  Management  Information  Support  System  (ARMISS) ,  Project 
No.  5RF-5025.  The  Agency  takes  exception  to  many  elements  of  the  report. 

Significant  amounts  of  documentation  and  other  pertinent  information  were 
provided  the  combined  DoD  and  DIA  IG  audit  team.  Every  effort  was  made  to 
provide  support  to  the  audit  team  in  providing  details  of  roles, 
responsibilities,  events  and  actions.  However,  the  audit  report  as  written 
does  not  reflect  a  large  part  of  the  information  provided  and  in  some  cases 
reflects  the  lack  of  interviews  of  key  individuals  (such  as  the  Chairman  of 
the  ARMISS  Source  Selection  Evaluation  Committee).  The  draft  audit  report 
needs  to  be  updated  to  reflect  the  correct  assimilation  of  events  in  context 
and  sequence.  These  include  the  long-standing  demand  for  an  Agency  management 
information  system  through  multiple  separate  contracted  and  Government 
examinations,  studies,  team  efforts  and  reports.  The  substantial  role  the 
representatives  across  the  Agency  played  in  developing,  defining  and 
validating  functional  requirements,  and  coordinating  throughout  the  entire  and 
extensive  process  leading  to  two  specific  contracting  efforts  (the  Request  for 
Information  (RFI),  and  Request  for  Proposal  (RFP)),  including  the  numerous 
corollary  actions  such  as  pre-proposal  conference,  bid  evaluation  plan 
development,  bid  evaluation,  successful  system  demonstrations  using  COTS 
products  by  the  bidding  vendors,  and  post-award  demonstrations- -and 
enthusiastic  reception  by  the  functional  representatives. 

It  should  be  noted  that  all  the  principle  DoD  regulations  governing 
acquisition  processes  which  were  cited  In  the  draft  audit  report  (and  to  which 
DIA  must  conform  as  reflected  In  DIAM  44-2  and  DIAR  65-17),  will  be,  in  large 
part,  superseded  by  a  revised  DoD  Directive  5000.1,  and  DoD  Instruction  5000.2 
integrating  the  5000  and  8120  policy.  This  draft,  currently  in  review,  is 
Intended  to  bring  DoD  policy  and  procedure!  into  line  with  the  Federal 
Acquisition  Streamlining  Act  (FASA)  of  1994.  The  major  themes  of  this 
revision  are  to  replace  'outdated  management  techniques  and  philosophies..." 
with  a  ’...management  process,  modelled  much  more  closely  than  before  on 
sound,  commercial  business  practices’  to.  among  others  ’emphasize  our  reliance 
on  commercial  products’  and  to  balance  responsibility  with  authority  which 
"dramatically  reduces  the  burden  of  mandatory  procedures  and  specifications, 
and  encourage  prudent  risk  management*,  and  ’reflect  the  importance  of  working 
as  cross -functional  teams."  Much  of  this  has  already  been  articulated  over 
the  past  four  years  by  senior  DoD  and  DIA  officials,  and  this  DoD  revision  is 
the  formal  expression  of  such  previous  guidance.  A  companion  document,  DoD 
5000. 37H,  ’The  NDI  Handbook,"  is  also  in  coordination  and  expected  to  be 
Issued  in  the  near  future.  This  will  provide  a  guide  for  acquisition  managers 
and  personnel  in  other  functional  areas  for  buying  commercial  and 
nondevelopmental  items.  The  handbook  relates  the  impetus  for  using  commercial 
items  and  cites, 

"In  defining  requirements  you  must  give  first  preference  to  the  use  of 
commercial  items  and  second  preference  to  the  use  of  other  types  of 
NDI.  This  Is  a  provision  of  the  Federal  Acquisition  Streamlining 
Act _ Consider  the  application  of  NDI  as  a  matter  of  degree  rather 
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than  an  all  or  nothing  propositi on.... Buying  and  using  commercial  and 
other  nondevelopmental  items  also  present  some  challenges  and 
departures  from  acquisition  'business -as -usual.’  For  example,  items 
...may  require  performance  trade-offs....’ 

The  planning  and  Initiation  effort  on  ARMISS  sought  to  adopt  and  follow  the 
many  policy  statements  that  early  in  the  1990s  embodied  those  ideas  and 
practices.  Though  DIA  followed  guidance  as  it  existed  at  the  time,  DIA  also 
attempted  to  follow  the  Government’s  intent  and  direction.  That  intent  and 
direction  has  subsequently  become  institutionalized  in  increasing  detail  in 
Government  and  DoD  policies  and  directives.  The  evaluation  of  this  ARMISS 
effort  must  be  in  the  context  of  these  drastic  changes  that  were,  and  still 
are,  taking  place  in  the  Government  and  DoD  acquisition  realm,  because  those 
efforts  were  in  consonance  with  the  guidance  that  has  been  consistently 
promulgated  by  senior  DoD  officials. 

Substantiation  of  the  concepts  upon  which  the  ARMISS  planning  effort  was 
founded  has  been  consistently  voiced  from  authorities  outside  DIA.  The  DoD 
Office  of  the  Comptroller,  the  Chief  Financial  Officers  Council  (composed  of 
the  Chief  Financial  Officers  of  federal  agencies  covered  by  the  Chief 
Financial  Officers  Act  of  1990)  and  the  Joint  Financial  Management  Improvement 
Program  (began  in  1988)  have  reiterated  precepts  fundamental  to  the  ARMISS 
planning  in  addition  to  OSD  C3I  Corporate  Information  Management  direction. 

In  line  with  the  above.  Agency  comments  addressing  shortcomings  in  the 
draft  audit  report  are  being  provided  as  follows. 

-  First,  there  will  be  general  comments  on  the  overall  draft  report  to  serve 
as  the  foundation  for  the  more  specific  detailed  comments. 

-  Then  detailed  specifics  of  the  report  are  addressed  page  by  page.  Each 
allegation  for  which  substantiation  has  allegedly  been  found  will  be 
addressed. 
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GENERAL/OVERALL  COMMENTS 


1.  The  overall  report  is  difficult  to  follow,  even  for  those  knowledgeable 
about  the  sequence  of  events  and  the  actions  taken.  The  report  does  not  give 
the  complete  picture  of  events  in  many  areas,  or  include  distinguishing  dates, 
thus  creating  serious  negative  impressions  of  actions  taken  or  not  taken. 

There  were  over  four  years  (1990- -1994)  of  activity  connected  with  the  ARMISS 
effort,  in  which  much  happened.  The  conciseness  of  the  report  does  not 
reflect  consideration  of  the  full  picture.  Material  is  included  below  to 
support  recommended  revisions/rewriting. 

2.  The  Agency  does  not  concur  with  the  draft  report  statements  regarding 
roles  and  responsibilities.  The  citing  of  an  "ARMISS  program  manager"  must  be 
corrected  throughout  the  draft  report.  Additional  material  needs  to  be  added 
to  reflect  understanding  of  the  tasking  sequence;  appropriate  terms,  roles  and 
responsibilities;  and  distinguish  between  the  effort  to  address  the  long¬ 
standing  problem  for  the  Agency  at  the  macro  level  in  the  planning  phrase,  and 
the  Implementation  of  the  AIS  strategy  developed  during  the  initial  effort  at 
the  project  level.  The  draft  report  must  be  corrected  to  recognize  the 
intrinsic  and  continuing  involvement  of  functional  and  organizational  team 
members  throughout  the  entire  process,  and  the  planned  and  structured 
Involvement  of  Agency  representatives  through  implementation.  The  gathering 
of  functional  requirements,  coordinating  with  Agency  representation  from 
working  group  to  executive  committee  presentation,  the  completion  of  all 
elements  of  the  Agency’s  acquisition  procedures  ending  with  uncontested 
contract  award  ana  the  creation  of  a  management  structure,  the  Administrative 
Functional  Control  Board  (AFCB),  intended  to  monitor  and  assist 
Implementation.  Once  the  contract  was  awarded,  an  AIS  technical  project 
manager  was  named  and  responsibility  for  the  contract  and  overall  ARMISS, 
including  COTS  customization  and  implementation,  came  under  the  purview  of  the 
assigned  AIS  technical  project  manager  in  consonance  with  Agency/SC 
procedures.  The  technical  project  manager  assigned  had  participated  in  the 
Request  For  Proposal  and  Statement  of  Work  effort,  was  a  member  of  the  ARMISS 
Source  Selection  Evaluation  Board  and  had  technical  qualifications  for  this 
complex  undertaking. 

The  leader  of  the  planning  effort  as  established  initially  by  senior  DIA 
management  to  set  the  direction  and  strategy  for  the  ARMISS  Project  worked 
with  Agency-wide  participation  throughout  the  effort.  The  same  individual  was 
also  designated  by  senior  DIA  management  as  the  leader  of  the  Process  Action 
Team  to  identify  and  make  recommendations  on  achieving  Agency  efficiencies. 
ARMISS  was  recognized  as  a  technological  improvement  that  would  indeed  achieve 
some  level  of  efficiencies,  but  that  true  efficiencies  stem  from  changes  in 
the  functional  processes  themselves.  To  ensure  management  control  of  both  the 
functional  and  technical  elements,  a  management  structure  was  proposed, 
approved  by  senior  management,  and  put  into  place  to  jointly  address 
functional  and  technical  Issues,  identify  and  propose  resolutions  to  conflicts 
in  priorities,  and  provide  oversight- -the  Administrative  Functional  Control 
Board. 
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3.  To  clarify  the  draft  report  regarding  what  was  the  ARMISS  concept  and 
strategy  and  what  was  the  implementation  control  and  monitoring  process  and 
what  individuals  were  responsible  in  each  phase,  the  following  is  provided: 

The  ARMISS  concept  is  simply  that  an  integrated  suite  of  applications 
meeting  the  major  administrative  functional  requirements  of  the  Agency 
provide  a  corporate  management  information  system  (MIS)  through  single 
data -entry  with  multiple  data  uses  at  all  levels  of  users  as  well  as 
higher  management  levels.  ARMISS  applications  would  interact  with  each 
other  in  a  way  that  would  appear  almost  seamless.  Continual  rekeying  of 
data,  non -value  added  redundancy  of  stovepipe  systems  and  manpower 
intensive  support  would  be  eliminated,  with  improvement  in  accuracy, 
responsiveness  and  demystifying  of  processes  with  shared  data.  ARMISS  was 
envisioned  as  an  ADP  "enabler"  to  achieve  efficiencies,  incorporating  OSD 
C3I  Corporate  Information  Management  principles,  policies  and  intent. 

This  included  the  use  of  commercial  off-the-shelf  software  products  to 
rapidly  achieve  functionality,  recognizing  and  accepting  a  less  than  100* 
solution  level.  This  approach  would  provide  the  corporate  management 
Information  system  functionality  as  recommended  by  previous  studies  and 
had  been  approved  by  the  participants  in  the  requirements  definition 
process.  It  was  briefed  to  senior  DIA  leadership  and  approved.  ARMISS 
was  designated  a  site-unique  migration  system.  These  points  appear  to  be 
missed  or  misunderstood  by  the  audit  team. 

The  ARMISS  contract  was  an  acquisition  effort  to  acquire  the  software  with 
innate  integratable  characteristics  that  met  the  functional  requirements 
of  the  Agency’s  Request  for  Proposal.  The  Oracle  suite  of  products 
acquired  Included  an  extensive  capability  to  be  tailored  to  specific  user 
demands  through  what  are  called  "flexfields."  These  "flexfields"  permit 
customization  without  changing  the  actual  code  structure.  However,  the 
customization  required  functional  users  to  both  understand  their  own 
process  and  be  able  to  articulate  reasonable  demands  of  the  data 
structure,  and  understand  the  capability  of  the  software  being  provided. 
The  ARMISS  contract  with  VSE  did  include  customization  of  the  software, 
module  by  module--or  function  by  function.  It  did  not  include  the  effort 
to  bring  that  software  into  the  Agency’s  environment;  that  was  the 
'  responsibility  of  the  Government  (DIA)  and  stated  in  the  Request  for 
Proposal.  Those  actions  were  intended  to  be  executed  through  in-house  or 
contracted  effort  using  the  pre-existing  DoDIIS  Integration  and 
Engineering  Contract  (DIESCON). 

The  ARMISS  implementation  was  the  Government-responsible  set  of  activities 
which  included  Contracting  Officer’s  Representative  (COR)  oversight  of  the 
VSE  contract,  plus  companion  activities  such  as  the  integration  of  the 
software  into  the  Agency,  and  ancillary  activities  falling  within  the 
parameters  of  the  ARMISS  concept  (such  as  the  Agency’s  Automated  Tasking 
System)  which  were  the  responsibility  of  the  AIS  technical  project 
manager. 

4.  The  draft  audit  report  overlooks  the  strategy  regarding  use  of  COTS 
products  in  implementing  the  ARMISS  concept.  The  announced  DoD  policy  was  to 
use  COTS  products  whenever  possible.  This  has  been  ever  increasingly 
emphasized,  particularly  in  functional  areas  in  which  private  industry  has  a 
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clear  lead.  The  argument,  reiterated  by  the  DISA  leaders  In  implementing  DoD 
migration  systems,  is  that  COTS  products  are  understood  to  provide  less  than  a 
lOOt  solution.  The  DIA  approach  considered  the  remaining  percentage  as  what 
should  then  be  the  basis  for  evaluation  to  determine  value-added  when 
discerning  and  evaluating  the  remaining  “customer  demands."  Private  industry, 
motivated  by  profit  margin,  has  led  the  way  in  developing  and  marketing  COTS 
products  whicn  are  highly  effective  and  efficient  in  reducing  manhours  for 
administrative  processes.  Per  OSD  and  DISA  guidance  (in  accordance  with 
Congressional  legislation  (1986))  “...requiring  DoD  to  give  preference  to  the 
acquisition  of  nondevelopmental  items’  use  of  COTS  is  stressed.  "Proof  of 
concept"  had  been  substantiated  by  the  responses  to  the  previous  Government 
issued  Request  for  Information  which  demonstrated  that  COTS  products  indeed 
did  exist  in  the  marketplace  which  could  be  used  advantageously.  Oracle,  a 
worldwide  leader  in  such  products,  was  cited  by  both  of  the  only  two  viable 
contenders  responding  to  the  Government’s  Request  for  Proposal. 

5.  The  report  also  confounded  the  two  DIESCON  delivery  orders,  both  with  each 
other  and  in  relation  to  the  ARMISS  contract  effort  by  VSE.  The  distinctions 
are  clear,  and  are  addressed  in  the  specific  comments  below. 

6.  It  is  mentioned  in  the  report  that  the  ARMISS  concept  and  SOW  stated  that 
the  Government  would  be  willing  to  change  the  ways  it  does  business  if 
necessary.  The  report  did  not  mention  that  upon  contract  award  this  initial, 
approved  strategy  was  not  followed  and  the  effort  was  allowed  to  go  in  the 
direction  of  seeking  a  traditional  from -the -ground -up  software  development, 
100*  solution.  Though  the  plan  for  the  technical  solution  had  been  arrived  at 
with  group  participation  across  functional  and  organizational  elements,  the 
implementation  itself  became  perceived  as  an  SC  effort- -not  a  multi-level 
functional  joint  effort  with  SC  providing  the  technical  support.  This 
transference  in  perceived  responsibility  from  group  dynamics  to  SC  led  to  the 
resumption  of  expectations  back  to  the  more  familiar  experiences  of  a  from- 
the-ground-up  development  effort,  resumption  of  expectations  for  an  ADR 
solution.  Instead  of  grasping  those  elements  of  the  software  which  did 
satisfy  requirements  and  would  have  resulted  in  shared  data  across  the  Agency 
and  multiple  levels  of  use.  demands  and  conditions  being  cited  as  absolutes 
stymied  tne  contractor  personnel.  This  in  turn  stymied  SC:  ADP  can  only  be 
an  enabler,  not  dictated  to  force  changes  In  functional  processes.  This 
important  point  and  its  contribution  to  the  decision  not  to  exercise  the 
contract  option  needs  to  be  incorporated  into  the  draft  audit  report. 

7.  DIA  complied  with  established  guidance,  policies,  procedures  and 
directives  and  intent  as  they  applied  to  the  COTS  product-based  effort  (e.g., 
DoD  Directive  5000.1,  section  c.  paragraph  l.c.,  "...maximum  practical  use 
shall  be  made  of  commercial  and  non -developmental  items"):  DoD  Directive 
8000.1,  "Defense  Information  Management  Program."  DoD  Directive  8120.1,  "Life- 
Cycle  Management  of  Automated  Information  Systems,"  DoD  Directive  5000.2, 
"Defense  Acquisition  Management  Policies  and  Procedures"  (part  5,  section  D. 
"Technology  Transition  and  Prototyping"),  DoD  Directive, 5010. 38.  "Internal 
Management  Control  Program. "  DIA  Regulation  65-17,  "Automated  Information 
Systems  (AIS)  Management  Policy,"  DIA  Manual  44-2,  "Acquisition."  Concepts, 
elements  and  goals  were  consistent  with  those  espoused  by  OSD  (C3I)  Corporate 
Information  Management  program  and  the  DISA  DoD  Enterprise  Integration  and 
Migration  Systems  efforts.  The  ARMISS  project  was  not  a  DoD  Standard  2167A 
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Software  Requirements  Development  Type  Effort.  None  of  the  contract  efforts 
addressed  in  the  report  were  put  in  place  to  do  a  ground-up  software 
development  effort. 

8.  DIA  has  kept  current  with  consistent  revalidation  of  the  precepts  which 
were  the  foundation  to  the  ARMISS  planning  that  are  evident  external  to  DoD 
from  the  Chief  Financial  Officers  Council  (CFOC)  and  the  Joint  Financial 
Management  Improvement  Program.  The  FAR  has  already  been  streamlined  in 
addressing  COTS.  The  drafts  for  revising  DoD  Directive  5000.1  and  DoD 
Instruction  5000.2  (superceding  DoD  5000. 2-M,  DoDD  5000.49.  DoD  7920. 2-M,  DoDI 
7920.4,  DoDD  8120.1,  DoDI  8120.2  and  numerous  policy  memoranda),  and  DoD 
5000. 37H  reflects  streamlining  and  addresses  COTS  consistent  with  the  DIA 
concept  applied  to  ARMISS. 
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DETAILED  SPECIFICS 

Draft  Audit  Report:  Executive  Summary 
Pages  i--ii 


Page  1.  Introduction. 

DIA  Response.  The  Report’s  over- conciseness  as  previously  stated  is  reflected 
in  the  Introduction.  The  compression  omits  key  elements,  and  rolls 
information  together  so  the  overall  result  is  Incorrect  In  regard  to  the 
entire  ARMISS  effort,  the  DIESCON  delivery  orders  and  their  relationship  to 
ARMISS.  and  the  functions  being  "replaced".  Recommend  rewording  for  clarity 
and  accuracy  as  follows: 

The  Defense  Intelligence  Agency  (DIA)  Intended  for  the  Agency  Resource 
Management  Information  Support  System  (ARMISS)  to  be  an  Integrated  suite 
of  commercial  off-the-shelf  (COTS)  computer  software  products  that  would 
enable  standardization  of  existing  adnini strati ve  systems,  elimination  of 
redundant  data  bases  and  rekeying  of  data,  promote  the  exchange  of 
information,  and  provide  a  common  data  base  of  resource  information  to 
support  management.  This  was  in  recognition  of  expressions  of  need 
provided  to  the  DIA  leadership  In  numerous  studies  and  after  a  DIA  Request 
for  Information  demonstrated  that  applicable  software  solutions  were 
available  In  the  marketplace.  In  June  1993,  the  Defense  Intelligence 
Agency  awarded  a  contract  to  VSE,  totalling  $6.9  million  to  acquire  COTS 
products  to  support  primary  functional  areas  (e.g.,  financial  management, 
personnel,  acquisition).  The  contract  also  included  services  to  be 
provided  by  VSE,  Inc.,  such  as  customizing  within  the  parameters  of  the 
COTS  products  to  meet  DIA  specific  functionality.  The  Government  was 
responsible  for  integrating  the  products  within  the  DIA  environment  and 
making  any  Interfaces  with  other  systems  or  products  as  necessary.  In 
September  1993  and  1994  the  Defense  Intelligence  Agency  issued  delivery 
orders  under  the  DoDIIS  Engineering  and  Integration  Support  Contract 
(DIESCON)  totaling  $1.1  million  to  bring  products  acquired  through  VSE 
into  the  DIA  environment.  The  DIESCON.  an  indefinite -deli very, 
indefinite-quantity  contract,  had  been  separately  and  previously 
established  for  integration  efforts  necessary  to  bring  ADP  products  into 
the  DIA  environment- -including  those  acquired  through  VSE. 

This  audit  was  performed  in  response  to  a  request  from  Senator  Strom 
Thurmond  to  review  allegations  received  anonymously  of  improprieties 
related  to  the  procurement  of  commercial  software  for  the  ARMISS. 


Page  1,  Audit  Results. 

DIA  Response:  DIA  non-concurs  with  the  statement  that  allegations  can  be 
substantiated.  Allegations  are  addressed  in  detail  below. 

Correction  of  this  section  and  the  rest  of  the  summary  in  consonance  with  the 
corrective  material,  comments  and  recommendations  provided  by  DIA  is 
requested. 
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Draft  Audit  Report:  Part  I  -  Audit  Results 


Page  2- -3,  Audit  Background. 

Page  2.  Need  for  An  Improved  Acini ni strati ve  Management  Information  System. 

DIA  Response.  This  paragraph  of  the  Report  is  overly  concise,  omitting  key 
facts  and  considerations  and  leading  to  misstatements  and  erroneous 
interpretations  and  conclusions.  Recommend  rewording  for  clarity  and  accuracy 
as  follows: 

Defense  Intelligence  Agency  (DIA)  administrative  systems  were  recognized, 
through  in-house  and  contracted  studies  and  evaluations,  as  fragmented, 
labor-intensive,  and  inefficient.  In  general,  administrative  information 
is  redundantly  processed  in  stand-alone  systems  that  do  not  provide  common 
access  and  were  developed  separately  for  individual  functional  purposes. 
The  necessity  of  cross-functionality,  efficiencies  now  drives  the  Agency 
toward  shared  data  for  a  multitude  of  good -management  reasons,  espoused 
throughout  DoD  and  private  industry,  for  example: 

treating  data  as  a  corporate  resource,  linking  functions; 
elimination  of  duplication: 

using  ADP  as  an  enabler  of  corporate  efficiencies  in  business 
processes,  not  the  solution  in  Itself; 
involving  the  functional  user  in  all  phase  of  activities: 
structuring  a  balance  between  functional  and  technical 
recognizing  that  incremental  solutions  are  necessary- -solve  the  big 
payback  Issues  first  and  work  on  the  rest  as  time  and  resources 
become  available. 

building  a  unique  system  from  the  ground -upwards  and  writing  code  is 
no  longer  affordable  in  development,  or  maintainable  in  upkeep, 
upgrades  and  training  costs; 

achieving  higher  accuracy  and  faster  response  time  because  the  data 
Is  not  being  hand  entered  repeatedly  with  differing 
meanings/timing/himian  error,  and  analytical  time  being  wasted  to 
track  disparities  in  reports  from  different  sources. 

In  July  1988,  the  Director  of  DIA  called  for  a  DIA- wide  management 
information  system.  A  team  representing  major  organizational  elements  was 
formed.  This  effort  evolved  in  1989  to  a  working  group  which  met 
regularly  and  held  the  objective  of  identifying  Agency-wide  applications 
Initially  for  integration  into  the  Agency  MIS;  provided  a  forum  for 
information  exchange  and  Identify  and  resolve  corporate  issues.  Issues 
included  integrating  Directorate  applications  into  the  corporate  MIS, 
procedures  for  managing  shared  data  in  corporate -wide  applications,  and 
corporate  versus  individual  user  funding/management  for  corporate 
application  development  and  implementation.  The  working  group  was 
broadened  for  better  functional  representation.  Unfortunately,  upon 
resignation  from  the  Agency  of  the  chairman  of  the  working  group  in  1990, 
momentum  was  temporarily  lost.  The  unfinished  objective  was  recognized 
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and  accepted  in  1991,  when  under  the  joint  leadership  of  DS  (now  SC)  and 
RS  (now  DA),  DS  accepted  chair  responsibilities  of  a  working  group  of 
representatives  from  key  Agency  organizations  to  bring  the  effort  to 
conclusion.  The  Chief  of  the  DS  Resources  Management  Division  was 
appointed  to  lead  the  planning  effort  which  was  essentially  a  continuation 
of  the  previously  established  team/working  group  mechanism. 


Page  2,  ARMISS  Concept  of  Operations. 

DIA  Response.  This  paragraph  of  the  Report,  as  written,  is  overly  concise  and 
is  missing  key  Information.  Recommend  rewording  for  clarity  and  accuracy  as 
follows: 

DIA  planned  to  establish  a  corporate  management  information  system 
supporting  multiple  users  with  an  integrated  set  of  software  tools  that 
provide  support  across  the  spectrum  of  administrative  functions  including 
Personnel.  Financial  Management,  Travel,  Training,  Acquisition,  and 
Logistics.  Technically  feasible  solutions  joined  the  then  fresh 
management  principles  of  Corporate  Information  Management  which  stressed 
ADP  as  an  enabler  of  management  efficiencies,  but  not  an  end  unto  itself. 
This  was  part  of  the  changing  DoD  culture  which  stressed  data 
standardization,  and  functional  drivers  to  technical  solutions,  systems 
consolidation,  software  standardization  and  reuse,  and  using  COTS  products 
instead  of  continually  reinventing  that  which  was  already  not  only  extant 
but  commercially  viable. 

The  goal  was  not  only  to  replace  multiple  stand-alone  systems,  each 
developed  separately  to  serve  the  organizational  needs  of  its  sponsor,  but 
to  provide  tne  Integrated  databases  to  functional  users  at  all  levels  of 
the  Agency.  Therefore,  for  example,  program  managers  at  every  level 
within  the  Agency  would  have  a  software  product  capable  of  providing  in- 
depth  information  about  their  resources,  not  just  Agency  budget  officials. 
These  integrated,  functionally  focused  sets  of  data  would  in  effect  create 
a  single  shared  database  available  at  varying  levels  of  access,  a 
corporate  management  information  system.  The  goal  is  consistent  with 
Corporate  Information  Management  policies  and  principles,  as  defined  in 
DoD  Directive  8000.1,  “Defense  Information  Management  Program,"  October 
27,  1992,  which  emphasized  the  capability  of  automated  data  processing  in 
support  of  business  process  Improvements,  data  standardization,  and 
systems  consolidation.  This  is  also  consistent  with  the  Joint  Financial 
Management  Improvement  Program  (JFMIP).  Principles  associated  with 
meeting  the  Corporate  Information  Management  goals  include  function 
process  improvement  projects  and  functional  technical  Integration  analysis 
and  planning. 

ARMISS  was  intended  to  promote  exchanging  and  sharing  information, 
minimizing  redundant  development  of  user  applications,  reducing  the 
potential  for  errors  by  eliminating  multiple  entries  of  the  same  data, 
maximizing  data  availability  to  upper  management  for  decision  making, 
maximizing  data  availability  to  lower,  multi-functional  echelons  for 
meeting  responsibilities  regarding  daily  operations,  status  reporting, 
situation  analysis  and  work  scheduling,  responsiveness  to  leadership  and 
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emergent  issues,  and  permitting  electronic  tracking  of  a  variety  of  data. 

The  Agency  estimated  that  commercial  software- -again,  in  keeping  with  OSD 
(C3I)  CIH  objectives- -would  provide  the  Agency  a  COTS  solution  that  would 
satisfy  the  majority  of  functional  requirements- -between  60*  and  80*.  In 
many  cases,  the  COTS  products  would  provide  users  with  capabilities  they 
never  had  and  would  otherwise  be  unlikely  to  get  in  any  near-term 
timeframe.  It  was  anticipated  that  interfaces  would  have  to  be  built  to 
major  systems  such  as  that  used  by  the  Agency’s  contracting  activity  and 
logistics  activity- -and  the  Agency's  accounting  activity.  Once  the  COTS 
software  was  operational,  review  of  the  remaining  individual  user  demands 
would  be  made,  assessing  the  value  added  of  that  demand,  with  whatever 
necessitated  it.  If  a  requirement  truly  withstood  critical  scrutiny,  then 
a  work  order  would  be  Initiated  for  either  Government  or  contractor 
solution.  The  key  to  ARMISS  was  that  the  software,  linking  the  major 
Agency  administrative  functions,  be  deliberately  Interlinked  from  the 
start,  giving  a  slngle-entry/multiple  function  character  to  the  effort, 
whatever  suite  of  software  was  used.  A  major  assumption  existed  then,  and 
remains  that  major  administrative  processes  do  not  functionally  vary  so 
substantially  within  Government  and  private  industry  to  preclude  a  COTS 
solution- -except  by  impediments  and  regulations,  and  processes  that  are 
within  reasonable  bureaucratic  control  to  change.  Therefore, 
considerable  effort  should  be  expended,  first  to  maximize  the  COTS 
solution  including  business  process  improvements,  and  then  to  take 
software  development  actions  only  when  proven  absolutely  vital  for  the 
good  of  the  Agency- -not  an  individual’s  user’s  preference. 

The  ARMISS  concept  was  a  direct  outgrowth  of  the  previous  studies, 
workgroups,  and  contracted/in-house  evaluations.  It  was  recognized  as  a 
technical  mechanism  to  achieve  some  improved  level  of  efficiency,  but  that 
true  efficiency  had  to  be  based  in  business  process  improvement.  These 
OSD  C3I  CIH  principles,  particularly  of  ADP  being  an  enabler  but 
necessitating  functional  management  for  direction  and  process  Improvement, 
were  substantiated  in  the  Agency’s  Administrative  Efficiencies  study.  The 
role  of  ARMISS  as  a  technical  improvement  was  substantiated.  It  was  in 
this  Administrative  Efficiencies  effort  that  the  ARMISS  planning  leader 
proposed  the  management  structure  that  was  put  into  place  in  the  Agency 
that  provided  the  discussion  and  decision- recommendation  forum  for 
addressing  functional  and  technical  issues  of  efficiency  achievement- -by 
automated  systems  such  as  ARMISS  or  by  functional/business  improvement 
efforts.  It  was  this  Administrative  Functional  Control  Board  (AFCB)  with 
representation  across  the  Agency  administrative  functions  plus 
organizational  representation  that  was  anticipated  to  play  a  major  role  in 
implementing  the  ARMISS  concept.  Functional  representation  included 
financial  management,  acquisition,  contracting,  logistics,  personnel, 
training,  travel.  Organizational  representation  included  the  major 
directorates  and  offices  such  as  the  Office  of  the  Comptroller.  Many  of 
the  participants  on  this  Board  were  also  very  active  in  the  subsequent 
acquisition  effort  as  well,  and  were  expected  to  provide  continuity  and 
focus. 

Contracting  for  Acquisition  of  ARMISS.  On  June  7,  1993,  the  Virginia 
Contracting  Activity,  DIA’s  contracting  authority,  awarded  contract 
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MDA909-93-C-0027  to  the  Value  System  Engineering  Corporation  (VSE)  to 
procure  and  customize  an  integrated  commercial -off -the  shelf  (COTS)  suite 
of  software  products  to  serve  as  the  foundation  for  ARMISS.  The  VSE  based 
its  bid  on  using  Oracle  Corporation  COTS  products  for  most  of  the 
administrative  functions:  financial  management,  acquisition,  personnel 
and  logistics  modules.  The  contract  was  a  cost-plus-fixed-fee  completion 
type  contract  for  procurement  of  the  products  and  customization  within  the 
scope  of  the  products  capacity  to  meet  OIA  specifications.  The  contract 
was  multiple  year  (5),  targeting  customization  by  functional  module  over  a 
two-year  period,  plus  maintenance  (following  three  years),  a  total  price 
overall  of  $6.9  million. 

The  contract  award  culminated  an  extended  period  of  requirements 
development,  articulation,  and  Agency  acquisition  efforts.  The  Agency  had 
assigned  responsibility  to  the  Information  Systems  Center  (addressed 
above)  and  appointed  a  planning  team  leader.  However,  the  entire 
acquisition  effort  was  characterized  by  Agency-wide  functional  and 
organizational  team  effort.  The  team  participated  fully  in  requirements 
determination,  definition,  then  developing  the  Statement  of  Work  for  the 
Request  for  Proposal.  They  also  coordinated  on  the  Agency’s  Advance 
Acquisition  Plan,  prepared  their  leadership  for  participation  at  the 
Agency’s  decision-making  forum  (the  Director’s  Executive  Committee).  They 
coordinated  on  every  acquisition  activity,  including  developing  the 
evaluation  plan,  demonstration  plan,  pre-bid  conference,  pre-award 
demonstration,  evaluation  of  bids,  and  post-award  demonstrations.  The 
team  membership  for  the  acquisition  effort  in  most  cases  overlapped 
membership  on  the  Actainl strati ve  Functional  Control  Board  (AFCB).  That 
Board,  as  proposed  by  the  ARMISS  acquisition  team  leader,  ensured  a 
management  structure  and  discussion  forum  to  address  both  functional  and 
technical  administrative  improvements  for  the  Agency. 

Under  terms  of  the  contract,  the  Agency’s  integration  of  the  ARMISS  COTS 
products  into  the  Agency’s  environment  was  the  Government’s 
responsibility,  to  be  executed  using  in-house  or  contracted  means.  The 
DoDIIS  Integration  and  Engineering  Contract  (DIESCON)  had  previously  been 
awarded  to  Computer  Sciences  Corporation  (CSC),  and  had  been  envisioned  as 
the  mechanism  to  achieve  this  integration  for  the  Government.  This  action 
was  initiated  as  DIESCON  Delivery  Order  #10  in  September  1993.  A  point  of 
factual  clarification  is  needed  regarding  Delivery  Order  #10.  The 
Delivery  Order  was  not  issued  for  $582,242,  but  for  $382,242.  Because  the 
Government  was  not  able  to  provide  Government  Furnished  Equipment  (GFE)  in 
a  timely  manner,  the  contractor  (CSC)  was  not  able  to  perform  in  the 
allotted  time.  Additional  time  and  funds  were  required  to  finish  the 
original  task,  necessitating  an  amendment  as  issued  12  July  1994  (signed 
by  Ms.  Rita  Le)  for  an  additional  $186,166.  This  brought  the  total  for 
the  task  order  to  $568,408.  Neither  the  correct  amount,  nor  subsequent 
date  was  cited  in  the  reprt.  At  time  of  initiation.  Delivery  Order  #10 
was  directed  at  establishing  a  test  bed  within  the  Agency  to  bring  up  the 
suite  of  COTS  products  in  their  original  (pre-tailored)  form.  Users 
across  the  Agency  would  be  able  to  use  the  test  bed  to  investigate  for 
themselves  the  range  of  functionality  provided  in  their  initial  form. 
Training  was  also  arranged  on  an  ad  hoc  basis  by  the  planning  team,  again 
for  familiarization  so  that  individuals  would  have  a  better  foundation  for 
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discussing  customization  efforts  with  VSE,  and  preparing  for  their  own 
data/system  conversion.  The  test  bed  would  reside  in  DS  (now  SC). 

The  VSE  plan  for  customizing  the  COTS  products,  concurred  in  by  the 
Government,  the  subcontractor  Oracle,  made  implementation  of  the  financial 
management  module  first  priority.  However,  because  of  the  delayed  date  of 
award  of  the  VSE  contract  (initially  targeted  to  January  1993),  the  Agency 
had  entered  the  end  of  the  fiscal  year,  and  the  support  of  the  Comptroller 
and  contracting  offices  were  not  available  due  to  workload.  Recognizing 
the  high  visibility  and  management  priority  accorded  to  problems  raised  by 
the  Personnel  element  of  the  Agency,  the  Agency  decision  was  made  to 
direct  actions  to  that  module. 

When  It  was  determined,  as  a  result  the  difficulties  with  the  ARMISS 
Personnel  module  work  by  VSE,  and  the  lack  of  participation  by  budgeting 
and  contracting,  that  an  integrated  product  was  not  going  to  be  achieved 
on  the  original  schedule,  Delivery  Order  #10  was  modified  to  focus  on  the 
SC  resource  management  system.  What  started  as  the  prototype  of  an 
integrated  Agency  corporate  information  management  system,  a  companion 
activity  to  the  VSE  customization,  the  modular  implementation  effort  was 
modified  to  provide  a  local  focus  expected  to  provide  benefit  to  the 
Agency  under  the  changed  circumstances. 

A  year  later,  DIESCON  Delivery  Order  #30  was  let.  This  action  was  in 
consonance  with  the  Director’s  continued  emphasis  on  process  improvement, 
and  Agency  efforts  to  implement  the  recommendations  made  by  Team  5  for 
Agency  ADP  acquisition  process  improvements,  addressed  at  the  Director’s 
Off -site  on  Process  (November  1993).  Team  5  included  the  principals  from 
the  Office  of  the  Comptroller,  Office  of  the  General  Counsel.  Production 
Center.  Collection  Center  and  Agency  Administration  Directorate,  chaired 
by  SC.  Customer  support,  coordination  streamlining,  tracking  and 
monitoring  of  acquisition  actions  and  access  to  that  data  by  the  customer 
were  among  the  process  improvement  goals.  To  provide  a  mechanism  to  help 
achieve  these  goals,  an  effort  to  implement  electronic  purchase  request 
coordination  (simple,  small  purchases- -below  $25,000)  was  initiated.  That 
effort  was  targeted  to  take  advantage  of  capabilities  in  the  existing  DOS- 
based  Acquisition,  Receiving  and  Inventory  Management  Information  System 
(ARIMIS).  That  effort  demonstrated  feasibility  of  the  concept,  but  also 
highlighted  the  need  for  either  investment  into  the  current  DOS-based 
system  to  make  it  robust  enough  to  fully  achieve  the  broader  improvement 
goals,  or  start  transition  of  the  application  into  UNIX.  Therefore,  in 
accordance  with  the  Agency’s  goal  to  transition  to  Unix,  DIESCON  Delivery 
Order  #30  was  let  to  Initiate  the  effort,  and  as  a  startpoint.  The 
Government  Furnished  Equipment  provided  included  the  Oracle  COTS  products 
already  acquired  through  the  ARMISS  VSE  contract.  If  ARMISS 
implementation  had  continued  as  scheduled,  the  current  separate  ARIMIS  as 
a  DOS-based  system  would  have  been  subsumed,  along  with  the  current 
separate  logistics/inventory  system  (ILS).  DIESCON  Delivery  Order  #30  was 
the  first  of  many  that  are  anticipated  as  needed  to  transition  the 
Acquisition,  Receiving  and  Inventory  Management  Information  System 
(ARIMIS)  from  its  DOS-based  application  to  UNIX  and  provide  the  mechanism 
for  customer  improvements  to  the  ADP  acquisition  process. 
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Work  also  continued  separately  on  the  training  module  in  conjunction  with 
NSA.  In  the  personnel  area,  two  things  happened: 

DIA  chose  to  adopt  the  Air  Force -developed  migration  system  for 
civilian  personnel  which  is  Oracle  based;  and 

Using  the  Agency- acquired  Oracle  relational  data  base  management 
system  (RDBMS).  DIA  is  developing  a  military  personnel  system. 


Page  4- -12,  ARMISS  Program  Management. 

Page  4.  ARMISS  Program  Management. 

The  report  says  that  DIA  did  not  exercise  effective  control  over  efforts  to 
acquire,  customize  and  Integrate  commercial  software  for  the  ARMISS. 

DIA  Response.  This  only  now  appears  so  in  hindsight  and  does  not  take  into 
account  what  took  place  and  what  were  and  are  real  management  Issues. 


The  report  criticizes  "...procurement  of  100*  of  the  assumed  quantities  of 
commercial  software  needed  for  the  ARMISS  without  verifying  that  the  system 
concept  design  was  feasible’ 

DIA  Response.  This  statement  includes  two  incorrect  perceptions.  Addressed 
one  at  a  time: 

"Procurement  of  100*  of  the  assumed  quantities...."  It  only  now  appears 
that  DIA  should  not  have  procured  the  number  of  licenses  that  it  did  in 
the  COTS  acquisition  through  the  VSE  contract.  At  the  time,  however, 
given  the  Agency's  acceptance  of  the  VSE  proposed  schedule  for 
implementation  within  two  years,  and  the  plan  to  get  users  involved  in  the 
products  through  the  test  bed  and  training,  and  the  very  good  price  being 
offered  on  the  licenses,  it  was  agreed  to  by  the  Agency  as  a  good  business 
decision.  Only  due  to  later  events,  when  the  ARMISS  implementation  plan 
radically  changed,  and  DIA  leadership  stopped  the  financial  management 
module  indefinitely,  and  implementation  went  into  hiatus,  that  the  number 
of  licenses  seemed  an  error.  Currently  over  200  of  the  500  licenses 
procured  under  ARMISS  are  being  used.  These  include  users  working 
training/Banner,  military  personnel  and  SC  resource  management  issues. 

"...verifying  that  the  system  concept  design  was  feasible."  The  notion 
that  the  system  concept  design  was  not  feasible  is  recurrent  through  the 
report,  demonstrating  continued  misunderstanding  of  using  COTS  products 
and  the  contracting  action.  The  functional  requirements  were  stated  in 
the  Statement  of  Work  of  the  Government’s  Request  for  Proposal.  The 
product  suite  proposed  by  any  bidding  contractor  was  to  be  integratable 
and  subject  to  proof  by  demonstration.  It  was  not  a  question  whether  the 
concept  of  using  COTS  was  feasible- -it  was  a  question  of  whether  the 
bidding  contractors  could  find  such  products  and  demonstrate  them.  Both 
of  the  two  viable  contractors  bidding  on  the  contract  did  so,  and  both 
used  Oracle  products  as  their  foundation.  Oracle  is  a  world  leader  in 
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these  administrative  COTS  applications  and  its  customers  include 
Government  as  well  as  private  industry.  Therefore  it  has  never  been  a 
question  of  whether  a  COTS  application  is  feasible,  but  rather  will  the 
DIA  users  accept  that  application,  recognizing  that  in  some  cases  100* 
satisfaction  cannot  be  achieved  for  the  entire  user  universe,  but  for  most 
others  at  least  a  60*  or  better  satisfaction  level  can  be  achieved.  This 
is  not  a  question  of  the  feasibility  of  the  system  concept  design,  merely 
leadership  commitment  to  accept  a  less  than  100*  solution,  and  willingness 
to  change  business  processes  to  maximize  the  usefulness  of  the  product 
applications. 


The  report  states  the  DIA  “...did  not  adequately  define  ARMISS  requirements 
before  contract  award,*  and  “...did  not  document  business  practices  to 
facility  customization  of  the  commercial  software....' 

DIA  Response.  Functional  requirements  were  identified  before  contract  award 
for  COTS  products,  and  functional /subject  matter  experts  had  cognizance  over 
their  functional  processes.  The  requirements  definition  was  requisite  within 
the  acquisition  process  of  the  Agency  and  the  contracting  process.  It  was 
only  arter  the  functional  user  community  individually  began  the  interview 
process  that  the  mass  of  “requirements"  and  allegedly  fixed  regulations  became 
apparent.  The  requirements  had  been  defined  functionally  and  it  was  expected 
that  any  appropriate  requests  for  refinement  would  be  detailed  in  the  meetings 
where  DIA  personnel  would  be  interviewed  by  VSE  personnel  as  part  of  the 
software  customization  effort.  These  functional  requirements  had  been 
coordinated  with  the  functional  area  expects  and  the  proposals  had  been 
evaluated  by  the  same  functional  area  experts.  The  rapid  prototyping 
methodology  utilizing  COTS  products  does  not  require  A- level  requirements 
specifications.  Functional  and  organizational  perspectives  had  been  well  and 
consistently  represented  throughout  the  entire  requirements  definition 
process,  pre-award  demonstration  by  the  vendors,  and  bid  evaluation. 
Representation  at  the  post-award  demonstration  was  emphatically  enthusiastic, 
and  Indicated  that  the  products  would  be  an  advance  over  what  was  currently 
available.  Without  any  actual  experience  with  the  software  itself,  users  in 
the  interview  process  created  massive  confusion  over  their  individual  demands 
and  individual  views  of  their  own  process.  As  for  Personnel,  for  example,  two 
previous  business  process  improvement  workshops  had  been  performed,  and 
conversations  with  functional  leadership  indicated  that  a  firm  understanding 
of  the  process  and  bottlenecks  to  efficiency  were  extant.  Likewise, 
functional  workshops  had  been  performed  on  the  acquisition  process,  and  though 
many  problems  and  criticisms  of  the  process  were  identified,  the  process 
itself  was  well  understood.  Actually,  the  Agency  permitted  slippage  in  the 
mandate  that  the  users  must  first  experience  the  product,  customized  to  the 
maximum  extent,  as  prerequisite  to  identifying  and  evaluating  what  additional 
requirements  were  left  “unsatisfied."  However,  the  circumstances  had  changed. 
For  SC,  or  the  Agency  leadership  for  that  matter,  to  continue  to  press  for 
user  acceptance  in  view  of  the  resistance  being  evidenced  In  the  Personnel 
implementation  effort,  the  result  might  have  been  an  implemented  software 
application  but  at  the  cost  of  morale  and  individual  esteem.  ADP  can  only  be 
an  enabler.  SC.  as  the  ADP  technical  center  can  only  propose  solutions.  If 
the  users  do  not  accept  the  solution,  regardless  of  how  technically  correct  it 
may  be.  or  how  much  more  it  offers  than  what  is  currently  and  anticipated  as 
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otherwise  available,  the  solution  will  not  work  without  greater  cost  to 
leadership. 


The  report  states  that  DIA  "...did  not  consider  the  requirement  to  rely  on  the 
National  Security  Agency  for  financial  and  accounting  services..." 

DIA  Response.  When  the  SOW  was  written,  the  Air  Force  was  the  Agency's 
disbursement/accounting  agent.  This  was  known  and  "considered"  in  the  SOW. 

The  Agency  recognized  that  it  would  have  to  feed  data  to  the  Air  Force.  This 
was  brought  out  in  meetings  with  OC-4  early  in  the  development  of  the  ARMISS 
SOW  and  RFP.  The  fact  that  NSA  replaced  the  Air  Force  does  not  alter  the  fact 
that  a  data  feed  to  an  outside  financial  and  accounting  function  was 
considered  and  deemed  feasible.  The  Agency  did  not  consider  the  NSA 
accounting  system  in  the  light  later  presented  by  the  Office  of  the 
Comptroller,  because  decisions  had  been--and  had  been  unchallenged- -in  the 
planning  stage  that  the  NSA  system  would  relate  to  the  Agency  similarly  as  the 
Air  Force  accounting  system  had  done  (and  the  Army  accounting  system  before 
that).  It  was  also  well  understood  that  the  OSD  migration  systems  effort 
could  affect  any  element  of  ARMISS.  but  that  decisions  regarding  those 
functional  areas  were  well  in  the  future  and  would  be  absorbed  or  adapted  as 
made.  As  an  internal -Agency  administrative  system,  it  was  anticipated  that 
interfaces  to  the  DoD  standards  might  be  required,  not  transition  of  the  daily 
operational  system  itself.  However,  when  the  Office  of  the  Comptroller  raised 
the  perception  later  that  the  financial  module  presented  a  redundant 
accounting  system  to  the  Agency's  singular,  NSA- provided  system.  Agency 
leadership  had  to  balance  organizational  rejection  with  that  of  a  technical 
solution.  It  had  never  been  proposed  or  considered  at  any  time  during  the 
planning  or  acquisition  phase  that  the  Oracle  financial  management  product 
would  perform  as  the  Agency’s  accounting  system.  It  was  only  intended  to 
provide  the  networked  venue  for  passing  and  applying  data  from  NSA  to  the 
various  functional  modules  for  daily  operations  at  a  sub -accounting  level  (in 
the  sense  used  by  the  Comptroller)  and  feeding  other  applications.  That  user, 
level  of  data  and  tool  availability  is  essential  for  the  Agency  to  continue  to 
move  in  the  direction  of  improved  efficiencies,  using,  for  example,  activity- 
based  costing  methods.  It  was  well  recognized  that  the  Agency  could  only  have 
one  accounting  system,  and  was  committed  to  Air  Force,  then  NSA.  However, 
technically,  the  Oracle  product  could  have  functioned  without  endangering  the 
legal  position  of  NSA’s  system,  and  provided  day-to-day  data  needed  by 
individuals  within  the  Agency  across  functional  and  organizational  levels,  but 
it  was  at  that  time  not  accepted  organizationally. 


The  report  states  that  DIA  "...continued  development  efforts  after  it  was 
apparent  that  the  acquired  commercial  software  was  not  suitable  for  the  ARMISS 
concept — " 

DIA  Response.  The  software  acquired  was  imminently  suitable  to  the  concept: 
the  applications  were  functionally  modular  and  Innately  integratable;  the 
proposals  by  both  vendors  cited  Oracle  products,  and  had  been  demonstrated 
prior  to  contract  award  as  part  of  the  evaluation  of  bids,  and  accepted  by  the 
Government  as  represented  by  the  Agency  team. 
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The  report  states  that  "...DIA  spent  more  than  $5.1  million  for  commercial 
software  and  contractor  customization  and  integration  services  that  did  not 
result  in  an  integrated,  management  information  system.’ 

DIA  Response.  This  statement  in  the  report  implies  that  VSE  contract  and  the 
DIESCON  Delivery  Orders  #10  and  #30  were  intended  to  produce  an  integrate, 
management  information  system.  This  implication  is  incorrect.  VSE  was  to 
provide  an  integrated  product:  Delivery  Order  #10  was  to  be  the  first  step 
toward  integrating  the  product  into  the  Agency:  and  #30  was  applicable  only  to 
ARIMIS- -not  ARMISS.. 


The  report  states  that  ’...continued  development  efforts  after  it  was  apparent 
that  tne  acquired  commercial  software  was  unsuitable  for  the  ARMISS  concept." 

DIA  Response.  The  origin  of  the  report  inference  Is  unclear.  The  COTS 
products  were  not  proven  unsuitable  to  the  concept  of  satisfying  a  demand  for 
functionality  across  administrative  functional  areas.  Rather  than 
"development",  efforts  to  apply  the  already  acquired  products  has  resulted  in 
successful  functional  operation.  SC  does  have  all  the  COTS  products  loaded: 
they  can  be  accessed  and  examined.  SC  has  succeeded,  through  using  the 
DIESCON  contractor  support  and  in-house  personnel,  in  making  the  financial 
management  product  operational  for  budget  purposes;  and  has  keyed  budget  data 
to  stages  of  functional  definition,  requisite  for  development  efforts  toward 
activity- based  costing.  In  addition  to  providing  the  management  tool  for 
every  day  budget  execution  activities,  an  additional  COTS  product  is 
operational  in  concert  with  the  Oracle  product.  This  provides  an  executive 
management  reporting  tool  as  well  as  analytical  device  for  the  working  levels, 
with  user- determined  level  of  detail  and  graphic  capability.  No  one  else  has 
loaded  or  attempted  use  of  the  Oracle  COTS  products  acquired  under  the  VSE 
contract.  Given  the  operational  capability  achieved  with  the  financial 
module,  there  is  no  foundation  for  the  flat  declaration  In  the  report  that  the 
acquired  commercial  software  was  not  suitable  for  the  ARMISS  concept.  Another 
effort  using  an  Oracle-based  COTS  product  is  also  now  operational,  supporting 
•Agency  training,  worked  by  DIA  in  collaboration  with  NSA. 


Page  4- -5.  Acquisition  Planning  for  the  ARMISS 

Page  5,  DIA  Time-Phased  Replacement  of  Existing  Systems.  The  report  states, 

"A  second  contract  would  be  awarded  to  Integrate  (perform  data  conversion  and 
download)  ARMISS  into  the  existing  DIA  environment." 

DIA  Response.  The  statement  is  Inaccurate.  DIA  planned  to  use  in-house 
personnel  and  contractor  support  as  appropriate,  in  keeping  with  the 
Government  responsibility  to  integrate  the  software  into  the  environment.  The 
pre-existing  Department  of  Defense  Intelligence  Information  System  (DoDIIS) 
Integration  and  Engineering  Contract  (DIESCON)  was  envisioned  as  the  contract 
vehicle  for  such  actions.  As  an  already  awarded  contract,  individual .delivery 
orders  would  be  written  and  approved  without  further  competition.  All 
delivery  orders  to  DIESCON  must  include  documentation  and  be  coordinated  and 
approved  through  the  Agency  acquisition  process  as  any  other  procurement 
action;  the  only  difference  is  that  no  competition  is  required  since  the 
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competition  already  took  place  in  the  contract  award.  SC  brought  the  SC 
financial  management  product  into  operation  using  a  combination  of  D1ESCON 
delivery  order  and  in-house  personnel  support. 

Page  5.  Prototyping  as  a  Means  to  Reduce  Risk  and  Uncertainty  In  the  ARMISS. 
The  report  cites  two  criticisms- -prototyping  to  reduce  risk  and  procurement  of 
500  user  licenses. 

DIA  Response.  The  ARMISS  contract  with  VSE  was  the  very  first  effort  to  apply 
COTS  products  to  achieve  a  system  solution.  It  was  very  definitely  not  a 
standard  from -the -ground -up  software  or  system  development  effort.  The 
availability  of  COTS  products  in  the  marketplace  meeting  the  administrative 
functionality  of  the  Agency  was  demonstrated  in  the  responses  to  the 
Government's  Request  for  Information;  this  was  considered  proof  that  a  COTS 
solution  was  feasible.  The  decision  to  use  COTS  products  was  in  keeping  with 
rapid  prototyping,  and  OSD  C3I  ClM  principles.  It  eliminated  a  multitude  of 
extended  activities  that  were  suitable  only  for  from-the-ground-up  development 
efforts.  System  concept  was  that  COTS  products  would  achieve  a  significant 
level  of  satisfaction  of  functional  requirements.  The  response  to  the  RFI 
indicated  that  such  products  were  on  the  market.  The  system  design,  which 
required  the  COTS  products  to  be  functionally  bundled,  was  proven  by  the 
responses  to  the  RFP  from  the  only  two  viable  contenders  for  the  contract 
award  and  demonstrated  by  both  prior  to  award. 

DIA  followed  the  life-cycle  management  and  AIS  policies  and  procedures 
dictated  in  the  DoD  Directives  8120.1  and  8120.2  in  establishing  a  strategy 
for  the  implementation  of  ARMISS.  DIA  sought  to  use  COTS  for  the  solution  of 
the  ARMISS  concept  as  required  under  DoD  Directive  8120.1,  paragraph  D.4.  In 
addition,  the  management  structure  established  sought  to  conform  to  paragraph 
D.9.,  minimizing  the  layering  of  management  by  having  the  project  manager 
report  to  the  NMISC  Director  for  technical  guidance  and  to  the  Administrative 
Functional  Control  Board  (AFCB)  for  Agency  functional  issues.  The  report 
(page  5)  addresses  acceptance  of  software  based  on  approved  test  results 
citing  DoDO  8120.1  and  8120.2,  but  these  references  apply  to  newly  developed 
software,  not  COTS  products.  COTS  products,  by  definition  have  already  been 
tested  and  are  operational  and  available  in  the  marketplace  as  commerci ally- 
viable  products.  Similarly  the  prototyping  policy  under  DoD  Directive  5000.2, 
part  5,  section  D  applies  to  newly  developed  software  and  is  not  applicable  to 
COTS  software.  The  purpose  of  the  system  "prototype"  planned  for  ARMISS  was 
to  provide  hands-on  experience  with  the  COTS  products. 

DIA  did  acquire  licenses  for  500  concurrent  users.  This  Agency  decision  was 
economically  sound  at  the  time,  given  the  pricing  offered  and  the 
implementation  schedule.  Given  the  acquisition  cycle,  if  implementation  had 

Eroceeded  as  scheduled,  perception  would  have  remained  that  it  was  a  good 
usiness  decision. 


Paoe  6- -8.  Technical  Constraints  Affecting  ARMISS  Implementation 

Page  6.  Docuaentation  of  Business  Processes.  The  report  ascribed 
responsibilities  to  the  ARMISS  program  manager,  and  assumptions  he  made 
regarding  the  detailed  documentation  of  business  practices. 
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DIA  Response. 

The  report  reflects  here  a  misunderstanding  regarding  "the  ARMISS  program 
manager"  leading  to  the  conclusion  that  the  effort  was  essentially  driven  by 
one  individual.  See  paragraph  2  of  the  DIA  GENERAL/OVERALL  COWEIfTS  section 
provided  above.  Efforts  were  the  result  of  group  participation  across  the 
Agency  over  an  extended  period  of  time,  and  at  least  using  three  levels:  that 
of  the  acquisition  effort  itself,  including  the  development  of  Agency- required 
documentation;  the  Administrative  Efficiencies  Process  Action  Team  and 
subsequent  Administrative  Functional  Control  Board;  and  the  Technical  Project 
Manager.  The  established  process  guided  the  project,  not  one  individual. 

It  was  the  Agency  decision  to  pursue  achieving  ARMISS  objectives  by  using  COTS 
products  rather  than  through  from-the-ground-up  software  and  system 
development  effort.  The  resources  and  time  required  to  approach  a  such 
solution  through  software  development  and  programming  were  not  available,  and 
duplicative  of  COTS  products  available  in  the  marketplace  as  demonstrated  by 
the  responses  from  industry  to  the  Agency’s  Request  for  Information.  It  was 
understood  that  using  COTS  products  would  not  result  in  a  100*  solution  to  all 
user  requirements,  but  that  the  functionality  provided  would  be  acceptable  as 
a  start  point.  Business  improvement  processes  such  as  evaluating  the  value 
added  of  any  user  requirements  not  met  by  the  COTS  products  would  be  applied 
in  the  decision  making  process  to  pursue  auxiliary  software  capabilities.  The 
acquisition  strategy  used  was  based  upon  rapid  prototyping  and  acceptance  by 
the  Agency  representatives- -both  functional  and  organizational --of  the  COTS 
solution  which  was  believed  as  60*- -80*  or  higher  solution.  Implementation 
required  functional  representatives  and  subject  matter  experts  to  understand 
enough  of  their  functional  process  to  participate  In  the  customization  effort 
available  within  the  scope  of  the  COTS  products.  Given  the  customization 
capacity  provided  by  the  Oracle  "flexfields"  and  the  recognition  that  not 
everything  that  all  the  users  would  be  achieved,  it  was  not  envisioned  at  any 
time  before  interviews  began  that  modification  of  the  underlying  programming 
code  would  be  addressed.  While  transition  of  inefficient  business  practices 
was  to  be  avoided,  this  implementation  was  not  intended  to  initiate  a  from- 
the-ground-up  software  development  or  business  reengineering  effort.  It  was 
intended  to  make  the  best  of  availabilities  provided  and  critically  assess  the 
remaining  customer  and  system  demands  in  context  of  the  best  interests  of  the 
Agency- -not  Individual  user  preferences. 

The  ARMISS  implementation  strategy,  incorporating  the  role  of  the  AFCB  and 
parallel  actions,  was  the  basis  for  the  functional  definition  of  requirements 
that  were  part  of  the  ARMISS  Request  for  Proposal  (RFP) .  The  statement  in  the 
report  that  the  "ARMISS  program  manager’  did  not  require  DIA  functional 
managers  to  document  detailed  definitions  of  their  functional  requirements  as 
required  by  DIA  Manual  44-2  is  incorrect.  In  accordance  with  the  concept  and 
implementation  strategy,  the  Agency  agreed  to  define  requirements  functionally 
at  a  functional  level  appropriate  to  COTS;  functional  managers  did  provide 
this  material  as  incorporated --and  approved- -in  the  Statement  of  Work  in  the 
RFP. 

A  third  assumption  needs  to  be  added  to  those  In  the  report  (page  6)  regarding 
the  implementation  strategy.  It  was  assumed  that  the  functional  users 
understood  their  own  business  processes;  it  was  this  knowledge  that  was  needed 
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to  customize  the  Oracle  "fl exfields."  This  assumption  has  a  very  sound 
foundation;  the  functional  managers  had  been  integrally  involved  in  every 
phase  of  the  planning  for  ARMISS,  development  of  the  Statement  of  Work,  and 
implementation  strategy:  the  knowledge  appeared  readily  available.  Although 
the  Personnel  managers  knew  what  they  needed  to  accomplish  their  function,  as 
submitted  in  the  RFP,  the  interview  process  with  VSE  reflected  an  insistence 
for  discrete  capabilities  in  excess  of  what  had  previously  been  understood, 
and  divergence  on  details  of  the  process.  This  was  in  spite  of  Personnel 
being  previously  involved  in  two  separate  business  process  improvement 
workshops. 

Page  6,  DIA  Undefined  Requirements  Affect  VSE  Ability  to  Perform.  The  report 
states  that  due  to  "...the  inadequate  requirements  documentation,  DIA  had  no 
assurances  that  the  Oracle  software  would  satisfy  the  ARMISS  concept... VSE  was 
unable  to  customize  the  Oracle  software _ 

DIA  Response.  It  was  the  change  in  stance  by  the  functional  user  groups  and 
acceptance  of  that  change  by  the  Technical  Project  Manager  that  provided  the 
opportunity  to  equivocate  to  the  point  where  VSE  was  no  longer  an  asset.  From 
the  acquisition  perspective,  the  Agency  did  everything  it  could  to  ensure  the 
best  effort,  including  contract  modifications  to  direct  the  contractor  to 
priorities  and  feasible  deliverables.  By  allowing  the  functional  users, 
through  the  interview  process,  to  expand  the  requirements  and  address  what 
were  believed  as  inadequacies  in  the  COTS  products  rather  than  the  level  to 
which  functional  operation  could  be  achieved,  the  previous  COTS-less-than-lOO* 
solution  dissolved  into  an  all  or  nothing  situation. 


Page  7,  Operational  Analysis  of  the  Oracle  Acquisition  Module.  The  report 
credits  VSE  as  identifying  that  interfaces  would  be  needed  for  external 
procurement  and  logistics  system,  and  intimates  that  the  import  interface 
would  have  to  achieved  by  modifying  the  Oracle  software. 

DIA  Response.  Two  major  ADP  systems  were  understood  from  the  initial 
acquisition  efforts  under  ARMISS  as  requiring  interfaces:  the  Virginia 
Contracting  Activity's  system  (SACONS)  and  the  Logistics  system  CILS).  This 
was  in  addition  to  the  understanding  that  an  interface  would  be  required  for 
the  Agency’s  accounting  system,  and  that  decisions  made  in  the  DoD  migration 
systems  arena  may  also  require  interfaces.  Extraordinary  effort  was  being 
made  at  the  time  to  download  and  upload  from  a  classified  to  an  unclassified 
environment  to  interface  the  DOS-based  SC  acquisition  system  (ARIMIS)  with 
SACONS  and  ILS  already,  so  the  Agency  had  considerable  experience  with  finding 
workarounds.  Again,  it  was  not  VSE  that  had  responsibility  to  integrate  any 
of  the  software  into  the  Agency  environment. 


Page  7.  Operational  Analysis  of  the  Oracle  Personnel  Module. 

DIA  Response.  Many  resources  were  devoted  to  attempting  to  implement  the 
Personnel  module  in  recognition  of  the  high  priority  given  to  personnel  ADP 
issues  which  was  done  to  accommodate  the  delay  necessary  in  addressing 
financial  management  and  acquisition  due  to  end  of  fiscal  year  priorities 
(then  later  dissolution  over  the  role  of  ARMISS  and  the  NSA 
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accounting/resources  management  capabilities).  However,  the  multitude  of 
demands  accumulated  through  the  interview  process  and  differing  views  of  the 
process  presented  confounded  VSE.  Again,  perspective  regarding  what  was 
supposed  to  be  a  COTS  solution  lost  focus  and  the  effort  devolved  into  what 
would  have  been  appropriate  for  a  from-the-ground-up  software  development 
effort.  Rather  than  focus  on  the  very  fundamental  understanding  of  what  a 
COTS  solution  means--less  than  a  100*  solution- -and  challenge  user  claims  as 
requirements,  or  address  why  suddenly  vital  and  unshakable  demands  appeared, 
asserted  as  somehow  mandatory,  VSE  took  all  the  input  provided  and  concluded 
that  the  COTS  personnel  module  could  not  satisfy  DIA  requirements.  The 
Government  had  recognized  in  the  planning  phase  the  fact-of-life  that  no  COTS 
product  could  be  expected  to  satisfy  100*  of  DIA  requirements;  however,  an 
acceptable  level  of  functionality  was  the  goal.  The  planned  Implementation 
strategy:  maximize  the  application  and  then  address  the  remainder  after 
implementation. 


Oracle  Software  Was  Not  Suited  to  ARMISS  Requirements. 

DIA  Response.  If  there  was  any  import  problem  Intrinsic  to  the  Oracle 
applications,  VSE  was  remiss  in  addressing  them  in  the  bid  proposal.  These 
specifications  were  clearly  stated  in  the  RFP  and  contract’s  Statement  of 
Work.  The  limitations  of  using  COTS,  and  the  need  to  update  any  interfaces  as 
the  COTS  product  evolved  were  understood  by  the  Government  and  accepted. 

Given  the  unacceptable  alternative  costs  and  development  time  needed  for  any 
in-house  effort  to  duplicate  even  that  level  of  operation  available  in  the 
COTS  products,  these  negatives  were  acceptable.  With  any  COTS- -or  GOTS  and 
internally  generated  code  product --interfaces  are  always  dependent  upon 
version  and  subject  to  change. 

As  for  DIA  not  adequately  evaluating  the  mechanics  of  how  the  Oracle  software 
worked  and  the  effects  it  had  on  the  ARMISS  concept  as  averred  in  the  audit 
report.  DIA  did  understand  that,  a)  the  suite  of  products  would  provide  the 
interfacing  across  functional  areas  required;  and  b)  interfaces  with  external 
or  In-Agency  unique  systems  would  be  necessary:  and  c)  if  the  Interfaces  might 
not  be  easy,  neither  were  they  any  more  impossible  than  dealing  with 
developing  interfaces  with  any  other  system.  Any  COTS  solution  would  incur 
these  or  similar  issues.  The  fact  that  other  private  industry  and  Government 
organizations  were  using  Oracle  products,  both  bidding  vendors  had  proposed 
them,  and  that  the  Statement  of  Work  had  been  clear  on  the  requirement  to 
interface  with  other  (including  external)  systems,  the  solution  seemed  in  the 
best  interests  of  the  Agency. 


Paoes  8-9.  System  Concept  Design  Incompatible  with  the  National  Security 
Aoencv  Finance  and  Accounting  System 

Page  8,  The  General  Ledger  Is  Key  to  Operation. 

DIA  Response.  In  the  context  that  DIA  intended  for  ARMISS,  this  is  absolutely 
true.  The  General  Ledger  Included  capabilities,  however,  which  may  have  been 
misperceived  as  duplicative  to  the  NSA  system's  role  as  the  Agency's  single 
accounting  system.  / 
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Requirement  to  Use  the  National  Security  Agency  System. 

DIA  Response.  In  the  report  there  is  mention  of  the  Memorandum  of 
Understanding  (MOU)  signed  by  the  Comptrollers  of  DIA  and  NSA  on  16  September 
1993.  There  is  also  a  statement  that  in  August  1993  the  DIA  Comptroller 
objected  to  the  use  of  the  ORACLE  budget  module.  However,  in  order  to  keep 
the  context  accurate,  other  facts  need  to  be  Included  In  the  report,  such  as: 
the  Contract  was  signed  in  June  1993;  the  Comptrollers’s  office  provided  a 
functional  area  expert  to  review  and  evaluate  bidder’s  response  as  well  as  the 
define  requirements  that  were  part  of  the  ARMISS  Statement  of  Work:  the  ARMISS 
Technical  project  team  attended  sessions  at  NSA  and  briefed  them  on  the  ARMISS 
approach  before  contract  award  and  before  the  MOU  was  signed.  The 
difficulties  mentioned  in  the  report  (page  9)  that  ARMISS  would  not  provide  a 
single,  integrated  solution  as  DIA  envisioned  was  not,  as  the  report  states, 
because  of  technical  difficulties,  but  because  a  policy  position  surfaced  by 
the  DIA  Office  of  the  Comptroller  after  contract  award. 

The  report  states  that  the  ARMISS  contract  Statement  of  Work  (SOW)  had  no 
requirement  that  DIA  use  the  NSA  accounting  and  finance  system.  This  is  true, 
but  omission  of  vital  facts  makes  it  misleading.  When  the  SOW  was  put 
together,  DIA  was  using  the  Air  Force  as  its  disbursement  agent  not  NSA.  In 
the  SOW  in  paragraph  4.1.1.  it  states  that  ARMISS  will  have  to  interface  with 
the  Air  Force  system.  The  NSA  system  replaced  that  of  the  Air  Force.  To 
fault  the  SOW  for  failure  to  mention  a  system  that  was  not  part  of  the  DIA 
environment  at  the  time  creates  the  impression  that  a  function  was  ignored 
when  In  fact  it  was  not.  There  was  no  indication  that  the  replacement  of  the 
Air  Force  system  with  that  of  NSA  was  going  to  functionally  or 
organizationally  affect  the  ARMISS  effort.  A  batch  process  using  an  interface 
between  the  two  could  have  been  considered  acceptable.  It  would  have  been  a 
vast  improvement  on  what  had  been  and  continues  to  be  available. 


Paoe  9.  The  ARMISS  Contract  was  Unable  to  Meet  Goals 

Page  9,  The  ARMISS  Contract  with  VSE  Was  Allowed  to  Expire. 

DIA  Restxmse.  At  the  time  that  the  decision  was  reached  not  to  exercise  the 
option  years  of  the  VSE  contract,  the  Agency  had  reached  a  standoff 
organizationally.  The  plan  regarding  Implementing  COTS  products  as 
coordinated  through  the  acquisition  phase  had  not  been  followed  in  technical 
project  implementation.  The  problems  being  raised  by  users  were  technically 
surmountable.  The  highest  levels  of  Agency  management  had  to  decide  whether 
to  dictate  the  ADP  solution  in  the  face  of  organi zati onal  1  y -expressed 
negativity  and  denial  of  support.  The  Agency  acted  conservatively,  for 
without  support  during  implementation  no  ADP  solution,  regardless  of  its 
purity,  can  succeed.  The  organizational  decision  was  to  permit  the  VSE 
contract  to  end  without  activating  the  option  years.  The  contract  was  not 
terminated  because  the  external  interfaces  were  too  hard  or  that  the  Oracle 
COTS  products  would  not  satisfy  the  ARMISS  goals. 


Page  10--12.  Project  Tracking  and  Oversight 
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Page  10.  Roles  and  Responsibilities.  The  report  identified  the  Chief  of  the 
Logistics  Division  as  the  ARHISS  program  manager  and  made  statements  and  drew 
conclusions  regarding  professional  conduct  with  VSE,  the  DIESCON  Delivery 
Orders,  and  senior  management  involvement. 

PI A  Response.  The  statements  and  conclusions  made  in  the  report  are 
inaccurate.  Each  will  be  addressed  below. 

The  DIA  Regulation  65-17  has  an  incorrect  date  cited  in  the  report.  The 
correct  date  of  this  document  is  6  November  1989,  not  6  November  1993. 

The  necessity  to  correct  the  'program  manager'  usage  has  been  addressed.  See 
paragraph  2  of  the  DIA  GENERAL/OVERALL  COMMENTS  ON  THE  DRAFT  REPORT  section 
provided  above. 

The  role  of  the  Chief  of  Logistics  Division  is  clarified  below: 

The  Chief  of  what  is  now  the  Logistics  Division  within  the  National 
Military  Intelligence  Systems  Center  was  designated  the  leader  of  the 
Agency-wide  effort  to  Identify  and  initiate  actions  to  achieve  an  Agency 
management  information  system.  This  directly  resulted  from  the  1991 
tasking  from  the  DIA  Director  to  what  is  now  SC,  stimulated  by  combined  DA 
and  SC  leadership  and  initiative.  With  only  the  direct  support  of  a  very 
small  core  team  within  his  own  Division,  the  Chief  coordinated  the 
planning  effort  ensuring  organizational  and  functional  representatives 
across  the  Agency  had  ample  opportunity  to  participate  in  actions  which 
led  to  the  ARMISS  concept,  gathering  of  functional  requirements,  and 
coordinating  the  effort  throughout  the  Agency.  Once  the  contract  was 
awarded,  responsibility  for  the  contract- -and  the  overall  project-- 
switched  to  the  Contracting  Officer’s  Representative  (COR)  and  the 
Technical  project  manager. 

The  Chief,  Logistics  Division  was  separately  tasked  by  the  Director  to 
head  an  Agency’s  Process  Action  Team  to  investigate  and  make 
recommendations  regarding  Agency  administrative  efficiencies.  This 
resulted  in  the  recommendation,  subsequently  approved  and  made 
operational,  to  establish  an  Administrative  Functional  Control  Board  and 
structure  to  address,  prioritize  and  discuss  technical  and  functional 
Issues  and  action,  as  well  as  make  recommendations  for  DIA  command  element 
decision  making.  While  it  was  envisioned  as  participating  in  the  ARMISS 
COTS  product  implementation,  it  had  a  larger  focus  than  just  ARMISS.  The 
initial  report  and  the  follow-on  Action  Plan  co-developed  with  DA 
recognized  the  ARMISS  contracted  effort  as  only  one  element  of  the 
Agency’s  effort  to  achieve  administrative  efficiencies. 

After  contract  award  action  the  Technical  Project  Manager  for  ARMISS 
(overall)  and  Contracting  Officer’s  Representative  (COR)  of  the  contract 
with  VSE,  were  designated.  The  Chief,  Logistics  Division,  assisted  in  the 
transition  into  the  implementation  phase.  This  included  efforts  which 
kept  the  Agency  leadership  and  interested  participants  informed  of  the 
progress  of  the  ARMISS  contracted  effort  through  issuing  electronic  mail 
updates  during  the  transition  period.  This  also  included  initiating  an 
ARMISS  Oversight  Committee  within  SC  for  reporting  by  the  project  manager 
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of  progress  on  the  contract  and  ancillary  activities,  both  for  Informing 
SC  management  and  as  a  discussion/decision  forum  for  addressing  problems 
encountered  and  anticipated  actions;  and  an  engineering  forum. 

Regarding  the  Contracting  Officer’s  letter  to  VSE  and  the  audit  report’s 
erroneous  conclusion,  correcting  clarification  is  provided  as  follows; 

The  DIA  contracting  officer’s  issuing  a  letter  to  VSE  reiterating  that 
only  the  contracting  officer  or  contracting  officer’s  representative  were 
authorized  to  issue  work  orders  seems  the  basis  for  the  erroneous 
conclusion  in  the  report  that  the  "program  manager"  was  interfering  or 
providing  guidance  to  VSE  inappropriately.  The  letter  was  not  directed 
personally;  inference  of  Inappropriate  behavior  is  subjective  and 
Incorrect. 

Regarding  the  implementation  and  integration  of  an  ARMISS-like  system  to 
satisfy  Systems  Center  Requirements,  DIESCON  delivery  orders,  correcting 
clarification  is  provided  as  follows: 

The  Chief,  Logistics  Division  has  responsibilities  regarding  the  SC  budget 
for  which  an  automated  tool  set  is  needed.  Recognizing  the  opportunity 
amidst  the  problems  resulting  in  the  VSE  customization  effort  and  changes 
to  the  implementation  plan  which  also  affected  the  initial  scope  of  the 
DIESCON  Delivery  Order  #10,  DIESCON  Delivery  Order  #10  was  modified  to 
include  using  real  budget  data,  provided  by  SC,  as  part  of  the  Agency  test 
bed  facility.  Though  not  really  ARMISS-like  (it  focused  on  the  financial 
management  module -budget  element  only),  this  maximized  the  usefulness  of 
the  DIESCON  Delivery  Order  #10  to  the  Agency  by  providing  direct  real 
advancement  toward  achieving  an  operational  budget  system  and  provided 
real  data  conversion  and  transition  experience.  As  an  integral  companion 
activity  to  the  VSE  customization  effort,  it  was  in  keeping  with  the 
Government’s  requirement  to  integrate  the  COTS  products  into  the  Agency 
environment.  The  benefit  to  the  Agency  was  that  this  led  to  SC  personnel 
being  able  to  make  the  application  operational  on  the  Local  Area  Network 
for  SC  for  budget  execution  Information,  along  with  an  executive 
Information  system,  and  link  the  budget  data  to  functional  definitions  for 
Activity-Based  Costing  efforts. 

The  report’s  criticism  of  the  program  manager  for  not  having  a  plan  to  track 
the  project,  and  not  reporting  status  of  the  ARMISS  or  DIESCON  efforts  to 
senior  management  are  erroneous  statements  and  stem,  evidently,  from  the 
misunderstanding  evident  in  the  report  as  to  definition  of  "ARMISS  program 
manager."  The  Individual  targeted  within  the  draft  audit  report  was  not  the 
ARMISS  technical  project  manager  or  COR  of  the  contract  with  VSE.  It  is  not 
Agency  policy  to  formally  brief  status  of  contracting  actions  to  senior  DIA 
management  unless  appropriate,  though  appropriate  concise  status  checks  are 
provided  through  weekly  key  components  meetings. 


Page  10,  DIA  Senior  Management  Briefings.  The  report  states  that  the  Systems 
Center  held  only  two  meetings  with  senior  DIA  management  concerning  the  status 
of  the  ARMISS  project  and  that  the  Chief,  Logistics  Division  initiated 
[DIESCON]  delivery  orders  without  proper  planning  documentation  and  senior 
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management  oversight.  The  report  states,  "In  our  opinion,  proper  planning  did 
not  occur  because  by  the  time  delivery  order  10  was  initiated  in  September 
1993,  the  DIA  Comptroller  had  reiterated  that  the  ARMISS  could  not  be  the 
final  authority  on  obligations  or  expenditures  and  could  not  provide  an 
integrated,  management  information  system  to  satisfy  DIA-wide  needs,"  and 
cites  the  VSE  perception  "...that  the  Oracle  software  did  not  conform  to  DIA 
business  processes  and  that  significant  risks  were  associated  with  required 
interfaces  with  external  systems." 

DIA  Response.  The  statements  and  inferences  in  the  report  are  inaccurate  and 
led  to  erroneous  conclusions  being  drawn. 

Regarding  the  report  comments  on  senior  management  involvement.  Agency 
authorities  such  as  DAP,  GC.  0C  and  SC  were  involved  in  working  problems  and 
acquisition  actions.  These  are  all  considered  Agency  senior  level  positions. 
As  stated  above,  it  is  not  standard  practice  to  regularly  brief  senior  • 
management  formally  on  all  on-going  contract  actions  unless  specifically 
requested  to  do  so  or  as  otherwise  appropriate.  With  the  many  and  varied 
contracts  in  the  Agency,  the  leadership  would  be  inundated  with  'for 
information  only"  material .  Concise  status  checks  are  usually  provided  to 
senior  management  of  significant  interest  items  through  leadership  or  key 
component  meetings  as  well  as  informal  meetings. 

The  statement  in  the  report  (page  11)  that  says,  "...the  ARMISS  concept 
continued  to  be  invalid  because  the  Oracle  budget  module  needed  to  interface 
to  with  the  National  Security  Agency  system  to  provide  an  integrated 
solution,"  is  incorrect.  First,  the  Oracle  COTS  product  suite  is  already 
integratable- -being  functionally  modular,  one  product  could  work  independently 
or  in  conjunction  with  its  other  functional  area  products  once  linkages  were 
activated.  To  make  it  fully  functional,  the  users  have  to  define  the  data 
fields  and  customize  the  "flexfields"  as  necessary.  Second,  the  concept  was 
not  invalid  due  to  the  concern  raised  by  the  Agency  Comptroller  that  this 
might  create  a  duplicate  accounting  system.  The  existence  of  COTS  products 
established  the  validity  of  the  concept.  It  was  policy  decisions  made  by 
senior  management  regarding  NSA  that  ceased  implementation  of  the  financial 
module,  not  because  the  concept  was  infeasible.  There  was  no  objection  to 
implementing  the  ARMISS  concept  at  the  Center  level --as  long  as  it  was  not 
implemented  at  the  Agency  level.  It  was  this  duplication  of,  and  competition 
with  the  NSA  system  which  covered  all  of  DIA,  which  was  the  issue.  Subsequent 
statements  by  the  Government's  Joint  Financial  Management  Improvement  Program 
statement  of  Federal  Financial  Management  System  Requirements  (FFMSR-0)  of 
January  1995,  "Framework  for  Federal  Financial  Management  Systems"  address  the 
single  accounting  system  issue: 

"The  financial  management  systems  policy  stated  in  0MB 
Circular  A- 127  requires  that  each  agency  establish  and 
maintain  a  single,  integrated  financial  management 

system _ Having  a  single,  integrated  financial  management 

system  does  not  mean  having  only  one  software  application 
for  each  agency  covering  all  financial  management  system 
needs.  Rather,  a  single,  integrated  financial  management 
system  is  a  unified  set  of  financial  systems  and  the 
financial  portions  of  mixed  systems  encompassing  the 


M  of  29  JAN  96 


24 


62 


Defense  Intelligence  Agency  Comments 


DEFENSE  INTELLIGENCE  AGENCY  COMMENTS  ON 
DoD  IG  DRAFT  AUDIT  REPORT  ON  ARMISS  (PROJECT  NO.  5RF-5025) 


software,  hardware,  personnel,  process  (manual  and 
automated),  procedures,  controls,  and  data  necessary  to 
carry  out  financial  management  functions,  manage  financial 
operations  of  the  agency,  and  report  on  the  agency's 
financial  status.... It  is  critical  that  financial 
management  system  plans  support  the  agency’s  mission  and 
programs... and... are  incorporated  into  the  agency’s  plans 
for  information  technology  Infrastructure  and  information 
systems  as  a  whole." 


Page  12,  Future  Direction  of  ARMISS.  The  report  states  that  the  "Chairman  of 
the  Configuration  Control  Board  for  the  Acquisition  System  (Acquisition. 
Receiving  and  Inventory  Management  System  (ARIMIS))  submitted  a  tasking  to 
COMPEX  Corporation  to  transition  the  data  in  the  Acquisition  System  to  the 
Oracle  data  base  system.” 

DIA  Response.  The  statement  is  incorrect.  There  is  not  yet  an  operational 
UNIX -based  replacement  for  ARIMIS.  COMPEX  was  asked  to  prepare  a  proposal 
looking  towards  a  transition.  This  was  for  planning  purposes,  intended  to 
gauge  the  impact  upon  COMPEX  of  any  such  transition.  COMPEX  was  not  tasked  to 
transition  anything. 


Page  12-  -13.  Smmarv  The  report  reflects  the  incorrect  and  incomplete  nature 
of  the  narrative,  and  resultant  conclusions. 

DIA  Response.  The  summary  needs  to  be  corrected  to  include  all  the  facts, 
analysis  of  the  events  in  context  of  the  established  Agency  processes  for 
acquisition  and  contracting,  DoD  C3I  CIM  policies  regarding  COTS  products  vice 
software  development,  and  realistic  problems  that  surfaced  in  implementation. 

In  regard  to  the  criticisms  citing  references  to  the  Office  of  the  Under 
Secretary  of  Defense  for  Acquisition  and  Technology’s  Program  Management 
Review,  the  Agency  followed  a  very  specific  acquisition  process  which  did 
indeed  require  senior  management  involvement  as  well  as  extensive 
participation  throughout  the  Agency  working  levels.  Requirements  were  well 
defined  for  the  rapid  prototyping  effort.  The  Agency  did  initiate  DIESCON 
Delivery  Orders  #10  and  #30  similarly  to  stand-alone  procurements  as  the 
report  cites  as .a  Program  Management  Review  recommendation;  the  only  element 
missing  was  the  competition  effort.  Those  Delivery  Orders  were  properly 
evaluated,  documented  and  management  oversight  assured;  the  COR  for  those 
efforts  has  an  extensive  file  providing  documentation. 


Page  13.  Recommendations  for  Corrective  Action  The  report  recommends  "...that 
the  Director,  Defense  Intelligence  Agency: 

1.  Cease  all  Agency  Resource  Management  Information  Support  System 
development  and  implementation  efforts  until  a  comprehensive  acquisition 
plan  developed  in  accordance  with  Corporate  Information  Management 
policies  and  principles  is  approved. 
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2.  Establish  a  system  of  controls  to  verify  that: 

a.  The  requirements  of  the  Defense  Intelligence  Agency  intended 
users  are  fully  documented  before  an  acquisition  plan  is  approved. 

b.  Procurement  is  limited  to  those  quantities  required  for  testing 
until  the  suitability  of  the  product  or  service  for  the  intended 
purpose  is  demonstrated. 

c.  Quarterly  status  reports  detailing  cost,  schedule,  and 
performance  data  are  provided  to  senior*  Defense  Intelligence  Agency 
managers." 

PI A  Response.  The  Agency  does  concur  with  the  recommendations  but  must 
comment: 


1.  All  ARMISS  efforts  have  ceased,  with  the  exception  of  the 
training/Banner  effort  being  performed  in  conjunction  with  NSA,  per 
direction  from  the  Director  of  the  Agency  pending  receipt  of  the  Audit 
Report. 

2.  These  actions  apply  additional  workload  and  structure  to  an  ADP 
acquisition  process  that  the  Agency  has  been  working  to  streamline  and 
make  more  efficient. 

a.  Procedures  already  exist  that  ensure  documentation  of  user 
requirements  are  part  the  acquisition  plan  and  included 
appropriately  in  purchase  requests. 

b.  Limiting  procurement  to  testing  quantities  until  the  product  or 
service  is  proven  is  appropriate  only  in  development  efforts.  This 
would  result  in  inappropriate  delay,  workload  and  additional 
expenditure  of  resources  without  significant  benefit  when  an 
established  standard  (Agency,  Intelligence  Community,  DoD  or  private 
industry)  exists,  or  COTS  product  is  verified  as  technically 
feasible.  Technical  feasibility  can  be  tested  by  requiring  the 
bidding  vendors  to  demonstrate  their  capabilities  in  accordance  with 
guidance  as  was  provided  in  the  Request  for  Proposal . 

c.  Senior  managers  are  apprised  concisely  of  the  status  of 
contracting  actions  that  are  of  significant  interest  either  due  to 
costs  and  sensitivity  during  routine  leadership  meetings.  Specific 
formal  briefings  take  place  as  requested.  The  effort  to  provide 
material  that  would  be  "information  only*  in  quarterly  reports 

-represents  a  workload  and  expenditure  of  resources  provides  no  value 
x added. 
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Part  II  -  Additional  Information 
Pages  16- -29 


Appendix  C.  Allegations  and  Audit  Results. 

Page  22--23,  Allegation  2.  The  report  states  that  substantiation  was  found 
for  this  allegation.  "Specifically,  no  formal  funding  plan,  no  formal 
acquisition  plan,  and  no  requirements  definition  for  the  ARMISS  project." 


DIA  Response.  The  Agency  non- concurs  with  the  rationale  regarding  the  cited 
substantiation. 

The  report  on  the  one  hand  states  that  DIA  did  not  have  a  formal  funding  plan 
and  that  senior  management  oversight  permitted  the  realignment  of  funds  for 
ARMISS  Including  contract  modifications,  but  follows  that  with  the  statement 
that  the  Systems  Center  did  have  a  formal  acquisition  plan  but  it  was  not 
sufficient.  The  process  used  for  the  ARMISS  contract  acquisition  was  in 
accordance  with  Agency  acquisition  procedures  and  ensured  extensive 
coordination.  The  effort  was  a  COTS  product  acquisition  with  some  services, 
not  a  system  development  effort.  The  procedures  included  developing  and 
coordinating  throughout  the  Agency  an  advance  acquisition  plan  (AAP)  for 
Agency  Command  Element  signature  as  dictated  by  the  Virginia  Contracting 
Authority.  It  included  the  funding  profile  anticipated.  The  purchase  request 
itself  included  the  documentation  necessary  per  Virginia  Contracting  Activity 
requirements,  including  the  Statement  of  Work,  and  was  also  coordinated 
throughout  the  Agency.  The  Agency  had  Issued  a  Request  for  Information  which 
demonstrated  that  ARMISS  could  be  initiated  using  COTS  products- -technology 
existed.  The  Air  Force  (NSA)  factor  was  indeed  considered  in  accordance  with: 
a)  other  external  Interfaces- -the  Statement  of  Work  identified  the 
requirement:  b)  recognition  of  the  Air  Force  (NSA)  as  the  provider  of  the  sole 
accounting  authority  for  the  Agency.  Data  was  to  be  passed  from  Air  Force 
(NSA)  to  be  used  to  update  the  ARMISS  and  provide  Information  to  multiple 
users  at  all  levels.  Realignment  of  funds  to  cover  emergent  issues  and 
contingencies,  including  modifications  to  contracts,  is  a  normal  event  in 
resource  management.  The  fact  that  Agency  senior  management  permitted  this  is 
not  evidence  of  DoD  acquisition  regulations  not  being  followed. 

The  decision  to  procure  100*  of  the  assumed  quantities  of  licenses  as 
addressed  above  was  a  good  management  decision  at  the  time  it  was  made, 
predicated  upon  the  Government  accepted  schedule  of  implementation  and  the 
price  being  offered.  That  implementation  problems  obviated  the  need  for  the 
number  of  licenses  is  not  evidence  of  DoD  acquisition  regulations  not  being 
followed.  Currently  the  Agency  Has  a  requirement  to  use  200  of  the  500 
licenses.  This  includes  people  working  the  training  (Banner),  military 
personnel  system,  and  the  SC  resource  management  issues.  The  number  of 
licenses  required  is  expected  to  continue  to  increase. 
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Page  23.  Allegation  3.  The  report  states  that  partial  substantiation  of  the 
allegation  was  found,  'The  ARMISS  Program  Manager  applied  undue  pressure  on 
the  ARMISS  evaluation  committee  members  and  contracting  staff  to  award  the 
ARMISS  contract.’ 

DIA  Response.  DIA  non-concurs  with  the  rationale  regarding  the  cited 
substantiation. 

No  definition  of  "undue"  with  accompanying  documentation  is  provided.  The 
sentence  implies  coercion  or  force  of  some  kind,  but  does  not  describe  what 
took  place  or  in  what  direction.  The  result  is  a  negative  impression,  but 
without  substantiation  other  than  what  appears  some  action  officers’ 
preference  for  more  time.  The  Agency  had  a  contracting  award  schedule  as 
worked  with  the  Virginia  Contracting  Activity  and  the  Contracting  Officer,  and 
was  also  trying  to  schedule  follow-on,  post-contract  award  activities.  The 
Evaluation  Board  members  may  have  preferred  having  more  time,  but  this  is  a 
universal  human  trait.  There  was  no  coercion  to  force  individuals  to  act 
against  their  professional  judgement  and  personal  will. 

The  statement  that  the  contracting  officer  suggested  recompeting  because  the 
bids  were  not  exceptional  though  they  were  acceptable,  and  expressed  concern 
regarding  the  contractor  tasks  is  not  evidence  of  substantial  concerns.  There 
is  no  documentation  that  these  were  shared  by  senior  leadership  in  the 
Virginia  Contracting  Activity.  If  so,  the  contracting  officer  would  not  have 
signed  the  contract. 


Allegation  4.  The  report  states  finding  substantiation  of  the  allegation, 

"The  ARMISS  contract  was  permitted  to  expire  because: 

the  Oracle  products  could  not  be  modified  to  meet  DIA  needs,  and 
•  the  contractor  could  not  perform  because  of  DIA  undefined  functional 

requirements." 

DIA  Response.  DIA  non-concurs  with  the  rationale  regarding  the  cited 
substantiation. 

The  report  cites  as  substantiation  that  the  Oracle  produces  did  not  readily 
conform  to  the  DIA  business  practices  due  to  regulatory  requirements,  lack  of 
functionality  or  design  features  Inherent  in  the  software.  The  Agency 
approach  in  implementing  the  COTS  products  had  been  to  tailor  the  software  as 
far  as  the  inherent  capabilities  of  the  software  permitted,  to  meet  specific 
user  needs,' including  changing  the  business  processes- -not  the  other  way 
around  as  the  report  is  presenting  as  substantiation  of  the  allegation. 

Once  Implementation  was  complete,  whatever  degree  of  user -stated  specific 
requirement  would  be  assessed  in  terms  of  value-added,  including  regulatory 
citations.  Functionality  would  necessarily  be  expected  to  be  less  than  100* 
overall,  but  for  many  users  the  functionality  would  be  100*  greater  than 
existed  considering  the  lack  of  capabilities  otherwise  available  to  them.  The 
design  features  inherent  In  the  Oracle  COTS  products  called  "flexfields" 
provide  extensive  customization  capability.  None  of  the  statements  regarding 
the  impossibility  of  the  customization  effort  had  been  empirically  tested  and 
proven.  DIESCON  Delivery  Order  #10  has  proven  that  it  is  possible.  The  issue 
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of  importing  data  was  not  the  decision  element  in  the  Government  action  to  not 
activate  the  contract’s  option  years. 


Page  5--6,  Allegation  5.  The  report  cites  finding  substantiation  of  the 
allegation,  "The  ARMISS  program  manager  spent  more  than  $1  million  in  an 
attempt  to  use  the  Oracle  commercial  software  to  satisfy  resource  management 
deficiencies  in  the  Systems  Center  operational  systems.” 

DIA  Response.  DIA  non-concurs  with  the  rationale  regarding  the  cited 
substantiation. 

Some  confusion  exists  in  the  report  regarding  the  DoDIIS  Integration  and 
Engineering  Contract  (DIESCON)  itself,  and  specifically  DIESCON  Delivery 
Orders  #10  and  #30.  The  report  erroneously  appears  to  believe  that  the 
DIESCON  was  awarded  for  one  reason  but  executed  for  another.  Regarding 
DIESCON  Delivery  Order  #10,  once  the  implementation  plan  of  ARMISS  changed, 
the  scope  of  the  effort  was  modified  in  accordance  with  Agency  procedures  to 
focus  on  achieving  real -data  load  and  functionality  from  the  test  bed  that  was 
of  benefit  to  the  Agency.  The  effort  did  help  migrate  SC  away  from  the 
obsolete  Wang  system  it  was  using  for  budget  execution  activities.  The  fact 
is  that  this  Delivery  Order,  as  modified,  has  been  successful. 

DIESCON  Delivery  Order  #30  was  the  initial  effort  by  the  Agency  to  prepare  to 
migrate  from  the  Inadequate  DOS -based  Agency  Acquisition,  Receiving  and 
Inventory  Management  Information  System  (ARIMIS)- -apparently  the  "Systems 
Center  Acquisition  System*  being  cited  in  the  audit  results--to  UNIX.  This 
action  was  prompted  by  concerns  to  improve  the  ADP  acquisition  process  and 
implement  improvements  as  recommended  by  Agency  business/functional  process 
improvement  efforts.  The  expected  delivery  from  this  effort  was  not  a 
replacement  of  ARIMIS,  but  an  initial  step  toward  UNIX. 

The  $1.1  million  spent  on  DIESCON  actions  were  in  accordance  with  Agency- 
approved  goals,  objectives,  and  with  prescribed  Agency  coordination.  These 
funds  were  spent  in  efforts  to  provide  improved  capabilities  in  areas  of  SC 
responsibility  and  in  turn  are  in  the  best  interests  of  the  Agency  and  the 
Government.  These  facts  do  not  substantiate  the  allegation. 


Page  25,  Allegation  8.  The  report  states  finding  substantiation  of  the 
allegation  that  'There  has  been  virtually  no  real  agency  oversight  of  the 
ARMISS  project.” 

DIA  Response.  DIA  non-concurs  with  the  rationale  regarding  the  cited 
substantiation. 

The  report’s  conclusion  is  not  supported  by  the  facts.  The  planning  and 
acquisition  efforts  Included  extensive  coordination  throughout  the  Agency 
among  functional  and  organizational  representatives  who  were  responsible  for 
keeping  their  higher  management  apprised  of  progress,  assumptions,  decisions, 
direction,  etc.  The  Request  for  Information,  acquisition  plan.  Statement  of 
Work  and  the  Request  for  Proposal,  etc.,  were  staffed  in  accordance  with  the 
Agency  procedures.  They  had  been  precoordinated  with  the  working  level 
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representatives  and  modifications  made  per  their  recommendations  as 
appropriate.  Post-contract  award  activities  included  electronic  mail,  key 
component  meetings,  meetings  and  taskings  with  the  Administrative  Functional 
Control  Board,  and  meetings  with  leadership  to  discuss  issues.  The  fact  that 
senior  management  elected  not  to  exercise  the  second  year  option  of  the 
contract  with  VSE  demonstrates  that  they  had  been  kept  aware  of  the  problem 
and  were  actively  involved.  The  management  structure  (Administrative 
Functional  Control  Board)  had  been  put  into  place  to  actively  participate  with 
the  technical  Project  Office  as  well  as  conduct  process  improvement 
activities;  it  was  the  forum  for  discussing  issues,  identifying  and  resolving 
conflicts  In  priorities,  melding  Implementation  actions  with  functional 
Improvement  process  actions,  and  making  recommendations  for  Command  Element 
decision-making.  ARMISS  implementation  did  not  fail  from  lack  of  management 
structure  to  provide  oversight  and  direction,  but  of  management  will  to 
implement  the  original  strategy  when  users  pressed  for  the  traditional  100* 
solution  characteristic  of  from-the-ground-up  software  development  efforts, 
not  characteristic  of  use  of  COTS  products.  Faced  with  the  unachieveable 
demands  of  the  customers  and  what  amounted  to  a  reversal  of  the  original 
implementation  plan,  the  only  alternative  would  have  been  dictatorial  decree. 
As  previously  stated,  once  into  implementation,  the  attitude  shifted  into  an 
■us6  the  users  against  “them"  the  ADP  systems  people.  Forcing  transition  and 
use  of  an  ADP  tool,  regardless  of  the  improved  functionality  to  that  user 
group  and  the  Agency  as  a  total,  was  not  deemed  acceptable  over  the  users’ 
objections. 
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